On September 5, 2024, AliPay unveiled its first AI doctor in human form: “Dr. Zhang Tao AI Agent,” at the 2024 Inclusion Conference on the Bund. The AI health manager was co-developed by the AliPay technical team and Professor Zhang Tao, the Director of PKU Smart Health Technology Engineering Center and a vascular surgeon at Peking University People's Hospital.
At the conference, Dr. Zhang’s team shared their experiences regarding the application of the AI Doctor Agent. The AI Doctor Agent was designed to provide accurate and human-like responses 24/7 online. When users need further consultation, they can seamlessly switch to interacting with Dr. Zhang Tao himself while the AI Agent will summarize the previous information and provide it to the real doctor. After diagnosis, the AI Doctor Agent can continue to assist patients in interpreting medical reports and sending medication reminders, providing all the healthcare services that users need.
Dr. Zhang Tao noted that previous digital technologies, epitomized by Internet Hospitals, mainly addressed the connection between doctors and patients. However, to some extent, this approach has burdened leading physicians as more patients turned to the best doctors. In contrast, the AI Doctor Agent addresses efficiency by screening patients with real medical needs, maximizing the value of leading doctors.
The Zhang Tao AI Doctor Agent is built on Ant Group’s self-developed Bailing Foundation Model, and based on the Foundation Model, the R&D team developed a three-layer data structure to further train the AI agent. The first layer consists of data related to vascular surgery gathered from extensive literature, clinical guidelines, and expert educational materials. The second layer is a personal knowledge base, containing authentic medical records and consultation dialogues provided by Dr. Zhang Tao. The third layer involves personalized fine-tuning, sourced from the revisions and annotations of the AI-generated content provided by Dr. Zhang’s team.
In the latest version, the AI Doctor Agent can not only send text replies to users but also convert the replies to Dr. Zhang Tao’s voice to strengthen personal features. Looking forward, the AI Doctor Agent will introduce a "phone call" feature, allowing real-time voice conversations with users, akin to speaking directly with Dr. Zhang. This function will be especially beneficial for senior users.
As the very firstreal-identityrepresentationbehind the AI Doctor agent, Dr. Zhang closely collaborated with Ant Group to combine Foundation Model, professional medical knowledge, and internet marketing. The innovative combination has significantly improved the efficiency, convenience, and intelligence of medical consultations, bringing the expertise of leading doctors to the general public in the form of AI agents, effectively advancing the process of medical intelligence and accessibility.
The PKU Smart Health Technology Engineering Center at Peking University Lin-gang International S&T Innovation Center will continue to deepen its collaboration with Ant Group and other partners in the healthcare sector. Guided by the principle of using technology to improve health and fulfill social responsibilities, the center is committed to developing more intelligent healthcare and health management products.
