Microsoft’s recent holiday advertising campaign for its AI assistant, Copilot, depicts a seamlessly helpful tool capable of managing everything from festive lighting to recipe conversions. However,a new report from The Verge calls into question the accuracy of those depictions,revealing importent discrepancies between advertised capabilities and real-world performance. Testing of the scenarios presented in the ad-which notably includes a cameo by Santa claus-found Copilot struggling with tasks involving visual recognition, practical application instructions, and even basic calculations. The findings raise questions about the portrayal of current AI limitations in marketing materials.
Microsoft has launched a new holiday-themed advertisement for Copilot, positioning the AI assistant as a universal helper for Christmas preparations. The 30-second television spot depicts users interacting with Copilot on their computers to tackle tasks related to festive lighting, cooking, decorating, and neighborhood regulations. The ad also features an appearance by Santa Claus.
Urmărește cele mai noi producții video TechRider.ro
– articolul continuă mai jos –
A recent article from The Verge examines whether the scenarios presented in the ad accurately reflect Copilot’s current capabilities. The publication recreated the ad’s prompts step-by-step and tested the AI assistant’s responses in real-world conditions.
One central example in the spot involves synchronizing holiday lights to music. In the advertisement, Copilot guides a user through a control interface on a website called Relecloud, resulting in lights that pulse in time with a song. However, Relecloud is not a real company, but a fictional entity used by Microsoft in various demonstration materials.
Microsoft maintains that the scenarios are not simulated. Nicci Trovinger, Windows marketing general manager, stated: “All Copilot responses are real responses that Copilot provided for the scenarios and questions presented, at a specific point in time. The responses were shortened for conciseness to fit within the spot’s duration, consistent with standard advertising practices.”
Practical testing yielded different results. When presented with a static image of the ad’s interface, Copilot attempted to indicate where to click, but misidentified elements on the screen. It highlighted nonexistent options and “saw” buttons that weren’t there. In other instances, it claimed to have highlighted areas of the screen when no highlighting occurred.
Similar issues arose when using a real-world application, Philips Hue Sync. Copilot initially recognized the application and provided correct instructions, but subsequently began pointing to nonexistent functions and directing the user to already configured settings. The visual cursor highlighting feature was also slow and inconsistent.
The ad also showcases commands like “Help me understand these instructions” or “Convert this recipe to serve 12 people.” In testing, Copilot struggled significantly with these tasks. When asked to assist with assembling an IKEA Kallax shelving unit, the assistant confused parts and misinterpreted step numbers. Identifying the components was inaccurate, making the guidance difficult to follow.
When converting recipes, Copilot correctly recognized the need to adjust quantities but failed to complete the calculations. In some cases, it prompted the user to finish manually. In others, it misinterpreted scaling options on the recipe website. When explicitly asked to generate a document with the recalculated ingredients, Copilot confirmed the request but did not deliver the result. This highlights the ongoing challenges of achieving reliable AI-powered assistance in everyday tasks.
Another example from the ad involves verifying homeowner association rules. Copilot is shown capable of analyzing a document and an image of an inflatable holiday decoration on a property. In testing, Copilot correctly identified the relevant rule but offered vague conclusions and avoided providing a definitive answer.
The ad concludes with Santa Claus inquiring about delays in toy production. Copilot’s response, attributing the issue to excessive hot chocolate consumption by the elves, is clearly intended as humor.
The analysis reveals a significant discrepancy between how Copilot is presented in marketing materials and its actual performance in practical use. Microsoft asserts that the ad’s responses are genuine, but testing indicates clear limitations in understanding visual context, providing accurate guidance, and completing requested tasks. The findings underscore the need for realistic expectations as AI assistants continue to evolve.
