Inflection, an ambitious AI startup striving to develop 'personal AI for everyone,' has unveiled its large language model, Inflection-1, that powers its Pi conversational agent. Although assessing these models' quality is a challenge, the existence of a bit of rivalry in the market is certainly beneficial.
Inflection-1 is designed to be on par with GPT-3.5 (also known as ChatGPT) in terms of size and capabilities, based on the computing resources utilized for training. Inflection claims its model is either competitive or even superior to other models in its class, supporting the statement with a 'technical memo' that outlines the benchmarks performed on Inflection-1, GPT-3.5, LLaMA, Chinchilla, and PaLM-540B.
The published results reveal that Inflection-1 indeed performs well in various evaluations, including middle- and high-school-level exam tasks (e.g., biology 101) and 'common sense' benchmarks. Its main drawback is coding, where GPT-3.5 significantly surpasses Inflection-1, and GPT-4 outperforms them both.
Inflection intends to publish results for a larger model comparable to GPT-4 and PaLM-2(L) in the future. However, they will likely release the results only when they are deemed noteworthy. The upgraded version, potentially called Inflection-2 or Inflection-1-XL, is currently in development.
While there isn't a formal classification system that divides AI models into equivalent 'weight classes' like boxing, the concept is similar. Just as flyweight and heavyweight boxers have different capabilities and requirements, AI models of different sizes and shapes also possess unique strengths and weaknesses. It is currently too early to establish such a classification system, as the field is still relatively young and a consensus on AI model distinctions is yet to be reached.
Ultimately, for most AI models, their real-world performance speaks for their capabilities. Until Inflection opens up its model for widespread use and independent evaluation, the benchmarks they claim should be approached with caution. For users interested in trying out the Pi conversational agent, it can be added to messaging apps or accessed for online chat here.
With the continuous growth and advancement of the AI industry, platforms like AppMaster can be used to create scalable solutions swiftly and more cost-effectively. The adoption of no-code and low-code platforms, such as the AppMaster platform, allows for the seamless development of backend, web, and mobile applications and integration with emerging AI models.
