#healthcare

Another dire example involves an algorithm called "nH Predict", used by UnitedHealth Group (the largest health care insurer in the U.S.) to determine the length of stays it would approve for patients in nursing homes and care facilities. In a class-action lawsuit filed in November 2023, the estates of the two named plaintiffs-deceased at the time of filing-alleged that UnitedHealth kicked them out of care too early, based on nH Predict's output, even as the company knew the system had an error rate of 90 percent. The court filing says that UnitedHealth used this system anyway, counting on the fact that only a tiny group of policyholders appeal such denials, and that the insurer "[banked) on the elderly) patients' impaired conditions, lack of knowledge, and lack of resources to appeal the erroneous Al-powered decisions." The families of the two plaintiffs spent tens of thousands of dollars paying for care that went uncovered by the insurer. Reporting from Stat News largely confirms the allegations in the lawsuit, namely that after acute health incidents, UnitedHealth aimed at getting elderly patients out of nursing homes and hospitals as fast as possible, even against the advice of their doctors. Moreover, when patients challenged denials, physician medical reviewers were advised by case managers not to add more than 1 percent of the prior advised nursing home stay. And case managers themselves were fired if they strayed from those targets. The executive in charge of the division controlling the algorithm stated on a company podcast, in a comically evil admission, "If [people] go to a nursing home, how do we get them out as soon as possible?"

— The AI Con by Alex Hanna, Emily M. Bender