Rakuten Symphony engineers have made a significant breakthrough by modifying open source code that opens the doors for broader large language model (LLM) use in telecom. They have ensured the compatibility of the industry-leading open source Milvus Vector Database with IPv6 networks - an industry-first innovation.
As part of its efforts to put AI at the core of its strategy through the Rakuten Group AI-Nization initiative, Rakuten Symphony has been exploring how to develop in-house LLMs - AI models that analyse vast sums of data to and generate succinct text summaries. A key part of this strategy is leveraging the best possible platforms to power Rakuten’s LLMs, including vector databases, specialized systems designed for managing and retrieving unstructured data through ‘vector’ embeddings and numerical representations that capture the essence of data items like images, audio, videos, and textual content.
Unlike traditional relational databases that handle structured data with precise search operations, vector databases excel in semantic similarity searches using techniques such as the approximate nearest neighbor (ANN) algorithm, which is crucial for developing applications across domains, and for addressing the challenges posed by AI and large language models like ChatGPT, such as understanding context.
Rakuten Symphony engineers identified the Milvus Vector Database - an open source database which is horizontally scalable - as their platform of choice for LLM use and developing and maintaining AI applications. Milvus is renowned as one of the most advanced vector database platforms for AI applications, but previously ran on IPv4 and with a limited environment and was not compatible with Internet protocol version 6 (IPv6), meaning it could not be used on IPv6 networks. Rakuten’s strategic pursuit of Open RAN, both in Rakuten Mobile and to other operators internationally through Rakuten Symphony, necessitates the use of IPv6 in pursuit of the highest possible standards.
IPv6 helps identify local endpoint systems on a computer network and route online traffic and is the most secure and technologically advanced standard, addressing the problem of IPv4 address depletion that has resulted from prolonged internet use worldwide. The Internet’s original architecture had fewer than 4.3 billion IPv4 addresses available, and this necessitated the shift to IPv6 as those addresses run out. Regimented guidelines come with the territory of IPv6- a more secure and technologically advanced layer.
The open source community had been previously unable to find a solution that would allow the Milvus Vector Database to be hosted on IPv6 networks, however, Rakuten Symphony engineers made changes to the most recent version of the Milvus Vector Database’s open source code that ensures it is now supported. After several iterations and updates, Rakuten Symphony subsequently succeeded in hosting Milvus on IPv6 and in its Tokyo Kiba Lab. Rakuten Symphony engineers have successfully delivered custom-built code to make the Milvus Vector Database more reliable, more distributed, and with higher performance.
The solution now enables Rakuten Symphony to power LLM use cases and Rakuten Group’s ‘AI-nization’ initiatives through Milvus, increasing Rakuten Symphony’s maturity when it comes to LLM use.
Rahul Atri, President, OSS Business Unit, Rakuten Symphony, said, “Rakuten Symphony is committed to the highest possible standards when it comes to responsible and effective AI use in telecom. Our engineers continually display their ingenuity in solving real-world AI challenges, and this translates into innovative, cutting-edge solutions that support operators around the world with their technological transformation.”
