Urology Pearls: When AI advice turns toxic

MADJAR
Jerry, age 60, arrived at the emergency department in a sour mood. He told the doctor, “My neighbor is poisoning me.” He said he had always been in good health and had never had a medical problem or psychiatric condition. The doctor examined him but found nothing wrong. His vital signs and neurological exam were normal. Blood tests, however, revealed several abnormalities: elevated chloride level, low phosphate level, and negative anion gap indicating an imbalance in his blood’s acidity.
Jerry was admitted to the hospital. He told his doctors he was following multiple dietary restrictions and was on a severely limited vegetarian diet. This was consistent with blood tests showing deficiencies in several micronutrients — vitamin C, B12, and folate. He also reported that at home he had been distilling his own water. He was very thirsty yet paranoid about the hospital’s water, saying, “The water isn’t just water. I know it. They say it’s purified but I can taste the chemicals.” These paranoid thoughts worsened after several hours at the hospital. He began hearing voices calling his name and seeing his daughter in his hospital room –even though no one was calling his name and his daughter lived in another state. Later, he attempted to escape but was brought back to the ward. He was held against his will for grave disability–an inability to survive safely without help.
The doctors administered antipsychotic medications, gave Jerry intravenous fluids, and corrected his electrolyte imbalances. They consulted the Poison Control Department and UpToDate — a constantly updated medical encyclopedia–and concluded that the most likely diagnosis was bromism, or bromide poisoning. It took several days for Jerry to begin feeling better. Prompted by his doctors, he recalled that in the weeks before admission he had noticed facial acne and small cherry-color spots on his skin. He said, “I felt tired all the time. I couldn’t sleep. I felt off balance like I was drunk or walking on a boat. And I was thirsty all the time, no matter how much I drank.” These were all signs and symptoms consistent with bromism.
It took three weeks for Jerry to recover. When he did, he shared his story with his doctors: after reading about the harmful effects of sodium chloride (table salt), he decided to self-experiment by eliminating table salt from his diet and replacing it with something else. Naturally for him, he sought advice on the internet –specifically from ChatGPT (version 3.5), the leading artificial intelligence tool at the time. It told him that chloride can be swapped with bromide. Jerry took the advice to heart and made the change.
That is how he landed in the hospital with an unpleasant case of bromism.
The researchers reporting on Jerry’s case–Drs. Eichenberger, Thielke, and Van Buskirk from the University of Washington–tried to follow in Jerry’s footsteps. They asked ChatGPT what chloride can be replaced with and received the same response, but with an important caveat. ChatGPT stated that, among other substances, chloride can be replaced with bromide. It did mention that context matters, but it failed to inquire about why the authors were asking the question. Chloride can indeed be swapped with bromide — for cleaning, for example — but not as a dietary component.
On a personal note, I’ve been using ChatGPT extensively. I use it in my office to search for solutions to challenging medical dilemmas, and at home to find answers to almost anything I want to know. It draws from vast databases and can integrate and analyze information rapidly and effectively. It does carry risks, though. As Jerry’s case demonstrates, one must present questions in the right context and keep a critical eye on the responses. Like a human advisor, ChatGPT can misunderstand a question and provide a misleading interpretation. It takes knowledge and common sense to know when to trust its responses and when to decline the advice.
This story is based on an article A Case of Bromism Influenced by Use of Artificial Intelligence, published in Annals of Internal Medicine: Clinical Cases. I made some minor changes to the storyline to make it more accessible to non-medical readers.