University of Tokyo: Building a citadel of reliable AI for the real world

Hironori “Rick” Kobayashi was the head of a company based in the U.S. state of Indiana when he first met Kenny Song, an up-and-coming American artificial intelligence engineer working for Google Brain, a deep learning research team at the technology giant.

The online meeting several years ago that matched the two — whose nationalities, backgrounds and ages differ — could not have happened at a better time: They both wanted to start a company involving artificial intelligence (AI), in Japan, as soon as possible.

Song moved to Tokyo in March 2020. Kobayashi finished his third stay in the U.S., after spending five years as an executive of a major Japanese trading house, and returned to Japan in June the same year. They established Citadel AI Inc. in December, only a few months after they met face to face, with Kobayashi as CEO and Song as CTO.

The Tokyo-based company develops and sells systems that automatically and instantly detect anomalies in AI, whose quality can easily deteriorate. Misrecognition and misjudgment by AI can inflict severe damage to business and lead to compliance and security breaches. Indeed, the credibility of AI is an emerging global issue. For example, the European Union has proposed a draft of the first-ever legal framework for AI, which addresses the risks in operating AI in the real world.

Citadel AI has already procured 100 million yen in seed-round funding and has recruited a multinational team of engineers. It has launched business projects with several companies, including Suntory Holdings Ltd. With its eye set firmly on the global market, the startup uses English as its working language.

Seasoned businessman meets a young, talented AI engineer

After studying electronic engineering and graduating from the University of Tokyo’s Faculty of Engineering in 1986, Kobayashi joined the major trading house that later dispatched him to head Japanese and U.S. companies operating under its umbrella.

While he was in the United States, Kobayashi was involved in the trading house’s investments in U.S. information industry startups. This experience prompted him to consider becoming an entrepreneur involved in AI someday.

Song, meanwhile, had become acutely aware about problems associated with AI, such as data drift — unexpected and undocumented changes to data structure — during his time as a product manager at Google Brain. Song, who had planned to “start my own thing” ever since he learned computer programming in his high school days, moved to Japan, where his girlfriend was living, and joined UTokyo’s Graduate School of Information Science and Technology as a researcher.

Stimulated by other startups at FoundX


Several startups were taking part in the program at that time. Each gave a briefing on its progress twice a week. “We learned how much progress other startups were making, which stimulated or put pressure on us in a good sense,” Kobayashi recalled. “If entrepreneurs are left to do everything on their own, they have difficulty staying motivated. This is why such incubation support is essential.”

In addition, Citadel AI was selected by 1stRound, another support program for startups provided by UTokyo Innovation Platform Co., Ltd. (UTokyo IPC), a wholly owned subsidiary of the university established to help jump-start business ventures. This enabled Citadel AI to prepare itself for fund procurement — in 2021, UTokyo IPC and ANRI, a venture capital firm, together provided a total of 100 million yen from their funds, which allowed the startup to recruit talented engineers.

Automatically detecting AI anomalies in real time

At present, Citadel AI offers two systems: One is Citadel Lens, which performs a “health checkup” on AI by stress-testing and indicating weaknesses in the form of scores; the other is called Citadel Radar, a kind of “bodyguard” that conducts real-world monitoring of AI around the clock and prevents misrecognition and misjudgment.


Citadel AI’s system for proof-of-concept acceleration and faster quality improvement
According to Song, the AI development and operation process is divided into three stages: Development of a model (a mechanism to automatically learn based on input data and generate outputs); evaluating the model; and real-world monitoring of data and model performance.

“After I moved to Tokyo, I decided to do research about security risks — reliability risks — of AI applications,” Song said. “Through this process, we found a pretty big opportunity in terms of what our company could provide: reliable testing and reliable monitoring of AI applications.” The first stage, or the development of a model, is relatively easy with open-source software, but the latter two stages are difficult to handle, according to Song. This means his startup is tapping into what is basically an uncontested market.

AI tends to misrecognize and misjudge if it “learns” from biased data. But humans find it hard to detect such anomalies. In conventional real-world monitoring of AI, engineers manually search for and identify anomalies, and then allow the AI to relearn by adding new, human-labeled data — a time-consuming and cumbersome procedure.

Citadel Lens and Citadel Radar automate this procedure. By being able to explain AI’s judgment standards, the Citadel systems can pinpoint the optimal time for AI to relearn and prevent any learning that would lead to a deterioration in its performance. The company’s systems are compatible with any AI model format, making them Japan’s first general-use platforms for such purposes.

Suntory Holdings, which has taken a stake in Citadel AI, uses Citadel’s products in several of its AI systems, including one to predict how pallets (a portable platform onto which products are placed for storing and moving) can be collected. Citadel has also tied up with Groovenauts, Inc., which offers a cloud-based service that makes it easier for users to harness AI technologies. Linking Groovenauts’ service and Citadel Lens helps users to objectively assess the quality of their AI model. Song says Citadel AI’s goal on the technological side is to improve Citadel’s products to handle different kinds of applications and different types of data.

Kobayashi envisions a future world in which AI will become “like water and air” to humans. “But AI is not almighty, and does require quality maintenance,” he said. In the nearly two years from his startup’s inception, Kobayashi has become increasingly confident in his business model and is determined to make inroads into the global market.