Hossein Rahnama

In recent years, this sci-fi concept has moved closer to sci than fi. In 2011, Ray Kurzweil speculated about the feasibility of uploading one’s personality and history to exist autonomously in the cloud. Since then, several “upload” projects have emerged such as the now-defunct MIT spinoff Eternime. Yet, Rahnama was the first to present a feasible architecture to achieve an approximation of virtual immortality via digital replicas.

Nine years later, Rahnama continues to pursue Augmented Eternity as a professor at MIT Media Lab’s Human Dynamics Group. The project has evolved to reflect new AI developments such as Large Language Models, as well as legal, ethical, and user experience questions. After research revealed that many people felt uncomfortable holding a digital séance with a replica of a loved one, Rahnama recast the “Human-Ai agents” behind the avatars as empathetic story-telling machines rather than chatbots.

More recently Rahnama has launched a related AI research project called “perspective aware AI,” which we explore here. The technology enables the creation of specialized Human-Ai agent surrogates based on our digital history, knowledge, and perspectives. While the new project lacks the sci-fi appeal of digital immortality, it may well have a greater impact on business and society.

Perspective Aware AI
Rahnama’s perspective-aware AI models can be fitted with different digital channels including chatbots and avatars that mimic the creator. Yet, the project is concerned less about creating a digital facsimile than with making the creator’s knowledge and perspective available when they are not.

“The models allow us to see the world through other people's lenses,” says Rahnama. “Instead of relying on generic LLMs, we can train our own personal models to lend out our expertise and help form a more transparent, decentralized collective intelligence.”

Instead of relying on generic LLMs, we can train our own personal models to lend out our expertise and help form a more transparent, decentralized collective intelligence.

In a sense, the concept reboots the old rules-based expert systems within a modern AI framework and with a focus on reasoning, audibility, and contextual awareness. The project builds on a multifaceted, multimodal AI architecture that incorporates concepts from neural networks, symbolic AI, graph-based AI, structural learning, and LLMs. The key component is the use of “causal graphs that can understand relationships between different nodes and entities in your life,” says Rahnama.

The graphs are used to create “chronicles,” Rahnama’s term for a focused index of semantic knowledge and perspectives. Users can develop multiple chronicles based on their different knowledge bases or life roles, training the models on texts, emails, images, X-rays, MRIs, customer service transcripts, and other public and private data. The chronicles are then instantiated as active Human-Ai agents, which tap into the chronicles and distributed data sets to achieve a goal. This could range from answering questions to accomplishing tasks on the Internet.

The platform’s multimodal support enables access to the Human-Ai agents via different channels. These could include a mobile app, web page, text or voice chatbot, video avatar, spatial environment, or holographic interface. “We are exploring the technological, ethical, and governance issues of building multimodal systems and are learning which channels and media are more relevant for different use cases,” says Rahnama.

From Celebrity Influencers to Financial and Medical Experts
In the coming years, Rahnama plans to introduce a version of the perspective-aware AI platform that can be adapted to a variety of business and personal use cases. One such example would be a celebrity influencer who creates chronicles that enable tourists to tap into their recommendations and perspectives about a travel destination.

Yet, Rahnama believes the greatest opportunity lies in “complex, regulated industries with a lot of historical structural knowledge such as medicine, law, or finance.” Here, the chronicles would combine the insights of an expert with institutional data, rules, and protocols. 

For example, a home finance chronicle might combine a financial expert’s perspective with the user’s aggregated financial data from banks, employers, insurance companies, and other sources. “You could run Generative AI models around that let you ask complex, hypothetical questions such as, ‘How can I retire at the age of 50? How can I buy that car or house that I want?’” says Rahnama. “Security is ensured because the chronicle does not include data, only the relationships of the trade or topic.”

Flybits, a startup launched by Rahnama to explore commercialization opportunities from his lab, already offers a white gloved customer service application for financial firms based in part on perspective aware AI technology. “Instead of being put on hold at a call center, you can activate a Human-Ai agent in the form of an avatar that knows a lot about you,” says Rahnama. “As you interact, it gets better at using historical data and models. At Flybits, we say we are advancing from personalization to participation. Instead of personalization algorithms deciding what you want to see based on your history, our algorithms let you converse or otherwise interact with them to create a service that is tuned to your needs.”

Rent an AI Lawyer
Perspective-aware AI can help solve a common supply and demand problem: Experts in high demand lack the time and attention to share their expertise at scale. Rahnama poses the example of a leading intellectual property lawyer who has customers waiting in line to pay $1,000 per hour for her advice. The lawyer could expand her business by creating consulting chronicles that would answer simpler or preliminary questions a la carte for far less.

Other applications include a doctor examining a medical dossier who needs to quickly decide on a prognosis or treatment: “Say he wants the perspective of a colleague, but it is unlikely he could reach them or that the expert would have time to review the case,” says Rahnama. “By using their chronicle, he could access the expert’s knowledge and biases to help make the decision. He could also call up the chronicles of other physicians to bring in different perspectives.”

The hypothetical attorney or physician could also create separate chronicles to offer their knowledge and perspective on cooking or car care or parenting. “People are very contextual,” says Rahnama. “I may have different social parameters as a father compared to as a professor or a musician. Our challenge was to capture this context and bias without requiring many manual inputs or expensive retraining of data.”

The Value of Bias in AI
We asked Rahnama why he is embracing bias when much of the current AI research is aiming to eliminate it. Indeed, why focus on the perspectives of flawed, biased human beings when we can query Generative AI agents that contain much of the world’s knowledge?

“Bias can help us use AI as a transparent decision-support tool rather than only as a decision-making tool,” he answers. “People are biased, so maybe some AI models should be too.”

Rahnama adds that it is difficult to remove biases from LLMs, and that the user typically has no idea where they came from. In contrast, perspective-aware AI is designed to reveal and verify sources of information and bias through risk and audit dashboards.

“One reason we are targeting highly regulated industries is that we can provide better explainability, verifiability, and audit trails than LLMs,” says Rahnama. “Regulated industries need to be able to tell stakeholders and customers how they arrived at a decision or piece of advice. LLMs suffer from their lack of verifiability, audit, and provenance. It all comes down to trust. We like to say that data is the new asset class, trust is the new currency, and AI is the new economy.”

Symbolic AI and an AI Solar System
Rahnama believes that the key to verifiability is symbolic, or causal, AI, which learns through symbols and concepts much like human toddlers do. The symbolic concept drove early AI development at MIT and elsewhere in the 1960s and 1970s, but never took off. AI researchers instead turned to neural networks, which learn by recognizing patterns.

Recently, Rahnama and other AI researchers have begun to combine symbolic AI agents with neural networks in hybrid solutions. The symbolic agent typically plays an advisory role to help the neural net or neural net-based LLM comprehend concepts such as object permanence, context, and a sense of self as an agent interacting with the world. Symbolic AI can also reduce training time by leapfrogging problems that a neural net would need many thousands of training examples to solve.

“Our approach uses symbolic AI to represent the world along with our behaviors and human traits,” says Rahnama. “Most AI models struggle with context, but our symbolic agent can explain context and perspective to the neural net and show how it can differentiate between and interpret the different sides of people. This enables the creation of AI graphs, which help to understand users in a broader and more contextual fashion.” The architecture also enables the model to better leverage the power of bias. “By using graph learning and structural learning, our system allows the user to capture context and become aware of the creator’s biases.”

Rahnama describes the architecture as a solar system. The application, which in a typical AI stack would sit on the highest level, instead plays the role of the Sun in the middle. The inner orbits are models “that are very sensitive and proprietary and need encryption,” says Rahnama. “This is where blockchain could play a role.” The middle orbits are home to symbolic/causal inference models, compute mathematical models, Bayesian models, and other neural net models.

The outer orbits contain the multimodal access interfaces and the LLMs. “We are translating components of the LLMs into vectors to integrate them into our tiny symbolic models,” says Rahnama. “The key lies in interconnecting these models in real time, which we do by turning our agent graphs into vectors that update the master contextual graph chronicle. In a sense, our data strategy is more important than our AI strategy. If you can figure out the relationships between data elements, it is easier to decide what type of AI you want to use based on the context.”

Rahnama’s team has also applied the platform to robotics, an example of what he calls multi-dimensional AI. “For an educational robot, you could create a chronicle with the perspective and values of the parents. As a child interacts with the robot, reciprocal value systems can be created that represent how the family communicates.”

One project goal is to develop a platform that enables the interaction of multiple chronicles. “Imagine that you are a student interested in the intersection of AI and medicine,” says Rahnama. “You could simultaneously activate a chronicle about each topic to form a new chronicle that you could use to pose cross-disciplinary questions.”

Eventually, Rahnama would like to enable the owners of Human-Ai agents to send them out to collaborate with other such agents semi-autonomously. “When a project is launched, the agents could get together and collaborate,” says Rahnama. “Then the human creators of the agents would come in to complete the project. This is another way that AI could be used to save time and augment our capabilities rather than replace us.”