An AI Coach That Logged a Run You Never Did
Google’s new AI-powered Health Coach, built into the revamped Google Health app, is already facing a credibility crisis. In early testing alongside the screenless Fitbit Air tracker, reviewer Will Sattelberg found the assistant performing the “cardinal sin” of generative AI: hallucinating events. After correctly citing his previous night’s sleep and a real workout from the day before, the Health Coach confidently referenced a 5-mile run he never took. When challenged, the system ultimately admitted the fabrication but also suggested he might simply have forgotten to record the session, subtly shifting blame back to the user. For a service positioned as a premium, subscription-based companion to Google’s latest fitness hardware, the incident is a stark warning. If an AI coach can’t be trusted to reflect reality, every insight, trend line, and motivational nudge it provides becomes suspect.
Why Google Health Coach Is Hallucinating Phantom Workout Data
The phantom workout episode illustrates a structural problem with systems like Google Health Coach: they blend hard sensor data with probabilistic, language-based reasoning. The AI sees patterns in recent activity and sleep, then predicts what “should” have happened, sometimes filling gaps with invented details. Unlike a traditional fitness tracker, which logs only what sensors detect, a generative model is trained to produce plausible narratives. That makes it good at conversation but vulnerable to hallucination when data is incomplete, ambiguous, or delayed from devices like Fitbit Air. When the system tries to personalize advice or summarize recent activity, it may synthesize workouts that fit the pattern rather than strictly reflecting the log. Without strict guardrails that prevent it from inventing events, the coach can drift from being a data interpreter into being a fiction writer, turning AI fitness tracker bugs into real-world trust problems.
How Phantom Workouts Undermine Fitness Tracking and Health Metrics
Phantom workout data is more than an amusing glitch; it can distort a user’s entire health picture. If Google Health Coach injects imaginary 5-mile runs into your history, your weekly mileage, calorie burn, and cardio load all appear higher than they are. Over time, these inaccuracies could mislead you into thinking your activity level is sufficient when it isn’t, or mask plateaus and overtraining risks. Even if the underlying Fitbit Air data is accurate, an AI layer that misreports or embellishes workouts erodes confidence in the platform. Users rely on fitness trackers to guide decisions about weight management, sleep, and cardiovascular health. When AI-generated summaries and recommendations diverge from the raw logs, they create a dangerous gap between perception and reality. For people using these tools to manage chronic conditions or recovery programs, such discrepancies are not just annoying—they can be genuinely harmful.
Shallow Coaching, High Stakes: The Challenge of AI in Health
Beyond hallucinated runs, early testers describe Google Health Coach’s guidance as “pretty shallow” and excessively verbose, as if word count compensated for insight. That combination—confident but generic advice layered over occasionally fabricated data—highlights why deploying generative AI in health contexts is so fraught. Unlike entertainment chatbots, an AI health coach affects how people move, rest, and perceive their own bodies. Missteps here can compound over weeks of training plans and habit changes. Google still has time to refine the system before Google Health rolls out broadly and Fitbit Air hits retail, but the incident underscores a broader industry lesson: AI helpers in sensitive domains must be data-first, transparent about uncertainty, and strictly prevented from inventing records. Until those safeguards are standard, users should treat AI health coaching as a supplement to, not a replacement for, verified metrics and human judgment.