Propose corresponding test methods and mathematical formulas for ai3 different IQ types

Foreword: This article is the latest research article of the intelligent laboratory on artificial intelligence IQ in the future. It mainly proposes that the intelligence level of the intelligent system will generate three different IQ types for different test purposes. For these three AI IQs, this paper also proposes corresponding Test methods and mathematical formulas.

In our research, we found that when humans discuss the level of intelligent development of AI, the needs and purposes are not the same, so there will be differences in the evaluation of AI IQ. The first purpose and demand is to judge the current AI system (or robot). Whether it transcends humanity intellectually, the second need and purpose is to understand how smart a smart product is when it comes to serving humanity, and how much it costs. According to this key difference, the future intelligent laboratory proposes that there should be three kinds of IQ in the AI ​​system, namely, general IQ, service IQ and value IQ.

0. Background

Along with 2016, AlphaGo defeated human Go champion Li Shishi, the rapid development of artificial intelligence around the world, and the spread of artificial intelligence threat theory, and the rapid development of intelligent products. Can artificial intelligence surpass humans? What is the level of intelligence of these smart products? Answering these questions requires a quantitative approach to testing the level of development of intelligent systems.

Since the introduction of the Turing test in 1950, scientists have done a lot of work for the evaluation system of artificial intelligence development. In 1950, Turing proposed the famous Turing experiment, using questions and human referee methods to determine whether a computer has the same intelligence. As the most widely used artificial intelligence test method, but the Turing test does not test the intelligent development level of Ai, just judge whether the intelligent system can be the same as human intelligence, and it is interfered too much by human factors, relying heavily on the referee and being The subjective judgment of the tester, so it is often claimed that the program passes the Turing test without being strictly verified.

On March 24th, 2015, a paper published by the Proceedings of the National Academy of Sciences (PNAS) proposed a new Turing test method "Visual Turing test", which is used to more computer image recognition. In-depth evaluation.

In 2014, Professor Mark O. Riedl of the Georgia Institute of Technology believes that the essence of intelligence lies in creativity. He designed a test called Lovelace 2.0. Lovelace 2.0's test range includes: creation of novels with virtual stories, poetry creation, oil painting and music.

In solving the problem of quantitative testing of artificial intelligence, there are two problems in various schemes including Turing test. First, these test methods do not form a unified intelligent model, and based on this analysis, distinguishing between intelligent Multiple categories. It is impossible to conduct unified testing of different intelligent systems including humans; the second is that these test methods cannot quantitatively analyze artificial intelligence, or only quantitatively analyze some aspect of intelligence, but how much of this system achieves human intelligence, development The ratio of speed to the speed of development of human intelligence is not covered in the above studies.

In response to these problems, the research team proposed that there are three kinds of IQs for the intelligent level assessment of intelligent systems, namely: the general IQ of the intelligent system, the service IQ and the value IQ. The theoretical basis, detailed definitions and evaluation methods of these three IQs will be elaborated in the following.

1. Theoretical basis: standard intelligent systems and extended von Neumann architecture

The evaluation of the intelligence capabilities of intelligent systems, including human and artificial intelligence systems, faces two important challenges: first, artificial intelligence systems do not currently form a unified model; second, between artificial intelligence systems and living organisms represented by humans. There is no unified model at present.

In response to this problem, in 2014, the Center for Virtual Economics and Data Science of the Chinese Academy of Sciences was also a member of the Future Intelligent Lab Research Team. Liu Feng, Shi Yong, Liu Ying, von Neumann Structure, David Wexler Human Intelligence model, knowledge management domain DIKW model system, etc. The "standard intelligent model" is proposed to describe the characteristics and attributes of the artificial intelligence system and human beings in a unified way, and any agent is regarded as a system with "knowledge acquisition, mastery, innovation and feedback".

Based on this model combined with the von Neumann architecture, an extended von Neumann architecture can be formed. Compared with the von Neumann architecture, this model adds innovative creation capabilities, which can discover new knowledge elements based on existing knowledge. And new rules, making it into memory for use by computers and controllers, and interacting with the outside through input/output systems. The second addition is an external knowledge base or cloud storage that enables knowledge sharing, while the external storage of the von Neumann architecture serves only a single system. The extended von Neumann architecture will play an important role in building the IQ of AI.

2. Definition of three different IQs in intelligent systems

2.1 Proposal of AI General IQ

Based on the standard intelligent model, the research team established the AI ​​IQ test scale, and in 2014 and 2016, the AI ​​IQ was performed on more than 50 artificial intelligence systems including Google, Siri, Baidu, Bing, and 6-year-olds, 12-year-old and 18-year-olds. test. From the test results, the performance of artificial intelligence systems such as Google and Baidu has improved significantly compared with two years ago, but it still has a certain gap with the 6-year-old children.

It should be said that the above AI IQ test is to solve the problem that AI can surpass human intelligence. This research is to treat each intelligent system including robots, AI software systems, humans, animals and other organisms as equal agents. The level of intelligence that is displayed in interaction with other agents in nature.

The definition of AI general IQ is as follows: Based on the standard intelligent model, in order to solve the problem of “evaluating the level of development of each intelligent system”, each intelligent system is regarded as an equal agent, and the intelligent evaluation score formed by the unified AI IQ test scale It can be called the AI ​​system's general intelligence general intelligence quotient (AI G IQ).

2.2. Proposal of AI service IQ

In practice, we found that except for a small number of AI systems, which are for scientific experiments, do not provide supplementary services for humans. Most other AI systems are manufactured to better serve human beings, and its intelligence is also mainly In the process of serving human beings, the higher the level of intelligence, the better the service for human beings.

In this case, if the AI's general IQ standard is used for evaluation, it is obviously different from the original purpose of the product being manufactured. This requires us to select service-related indicators for evaluation based on the characteristics of such AI systems based on standard intelligent models.

These indicators are related to the AI's general IQ benchmarks, but there are large differences. Including the laws of artificial intelligence, ethics and other constraints should also be placed in the intelligent system service IQ. It doesn't have to be in the general IQ of intelligent systems.

The definition of AI service IQ: Based on the standard intelligent model, in order to answer the question “How does the intelligent system serve the human being better”, the intelligent level reflected by the intelligent product in the service process is tested, and the intelligent evaluation score is formed. It can be called the service intelligence of the AI ​​system, Artificial intelligence seveice intelligence quotient (AI S IQ).

2.3. AI value IQ proposed

AI systems that provide services or supportive work for humans are often provided by different companies and enterprises. For example, smart speakers include brands such as Amazon and Baidu. Smart chat robots include Keda Xunfei and Apple Siri. It is manufactured by different companies, and the same or similar functions are completed. The cost or price of each enterprise will be different. The relationship between service IQ and cost or price will have an important impact on consumers' purchase of smart products.

The definition of AI value IQ: Based on the standard intelligent model, in order to help users judge the intellectual ability of the intelligent system to obtain the economic cost, the intelligence evaluation score of the intelligent system divided by the selling price of the system can be called For the value intelligence of the AI ​​system, Artificial intelligence Value intelligence quotient (AI V IQ).

3. Intelligent system universal IQ and service IQ test scale design.

3.1. Intelligent System General IQ Test Scale

In order to solve the problem that AI can surpass human wisdom, in 2014, researchers in this paper divided intelligence into four categories: “knowledge acquisition, mastery, innovation and feedback” according to the standard intelligent model. Under these four categories, it was divided into 15 categories. A small classification ability to measure AI from more dimensions, human intelligence. These 15 small categories are: image, text, sound recognition and output, common sense, calculation, translation, arrangement, creation, selection, guessing, discovery, etc. Each small classification has different weights.

In 2017, based on the development of artificial intelligence and the latest research on intelligence. The research team adjusted the AI ​​general IQ scale from the test classification and classification weights. The main adjustments were added: 1. The ability to identify dynamic images, 2. The ability to recognize and express emotions, 3. The ability to identify the enemy and the enemy, 4 The ability to disguise real intentions, 5. the ability to achieve mobile positioning, 6. the ability to transform the world. In addition to this, the test of common sense and creation has also been done in a more detailed way.

The general IQ of the intelligent system is IQAIG, FGi is the secondary evaluation index item score, WGI is the weight of the secondary evaluation index item, and N is the number of evaluation indicator items. Therefore, the general IQ formula for an intelligent system is as follows:

3.2. Intelligent System Service IQ Test Scale

There are a large number of intelligent systems, such as chat bots, intelligent search engines, smart speakers, smart phones, smart cars, smart washing machines, smart refrigerators, etc., most of which serve as a commodity to serve human needs, these intelligent systems can Called smart products.

Under the standard intelligent system and the extended von Neumann architecture, they refine their common intelligent features, and according to different service requirements, form the following test scale of intelligent system service IQ. In the test IQ of this service IQ, the following aspects are highlighted.

1. Ability to sense the surrounding intelligent systems and user identities

2. Ability to interact with the Internet cloud

3. Display the internal status of the user in real time to the user, and the ability to provide support for failure

4. Ability to serve humanity in accordance with local laws and ethics

5. Ability to protect users and others in hazardous situations

6. Self-energy use and automatic replenishment capabilities

The service IQ of the intelligent system is IQAIS, FSI is the score of the second-level evaluation index item, WSi is the weight of the second-level evaluation index item, and N is the number of evaluation index items. Therefore, the service IQ formula of the intelligent system is as follows:

As a standard scale for intelligent product service IQ, in order to cover all kinds of smart products as comprehensively as possible, when designing the intelligent product service IQ test scale, the knowledge acquisition, mastery, innovation and feedback in the test scale are four. Aspects leave an interface with smart products:

The “other” information input method has been added to the knowledge acquisition classification to evaluate the new way of intelligent product input in knowledge.

In the mastery of knowledge, “professional common sense” has been added to evaluate the professional skills of smart products in different fields.

In the output of knowledge, additional output capabilities are added to evaluate new ways in which smart products are exported.

3.3 How to form AI value IQ

According to the definition of intelligent system AI value IQ (AIVIQ), if the intelligent system becomes a product to serve humans through sale, the service IQ of the intelligent system is AISIQ, and the open price of the smart product is P, forming an intelligent system value IQ. The formula is as follows:

IQAIV=(IQAIS/p)*100

4. Summary

According to the three IQs of AI, the intelligent system can have three different intelligent level evaluation methods and three kinds of IQs derived from the different usage and evaluation goals: AI general IQ, AI service IQ and AI value IQ. AI's versatile IQ has been thoroughly researched in papers since 2014. It also analyzes the differences between Google, SiRi, Baidu and other human IQs through the joint evaluation of AI systems and humans.

The newly proposed AI service IQ and AI value IQ provide theoretical analysis and implementation methods for evaluating the intelligence level of intelligent products. The follow-up work will be based on the AI ​​service IQ scale, for specific smart products, such as smart speakers, smart phones, smart cars, smart washing machines, smart refrigerators, etc., to carry out their AI general IQ, service IQ and AI value IQ evaluation work. .

The Future Intelligence Lab is a joint research institute of artificial intelligence, Internet and brain science jointly established by artificial intelligence scientists and related institutions of the Academy of Sciences. Author of Internet Evolution, Dr. Liu Feng, Ph.D., and Professor Shi Yong and Liu Ying from the Center for Virtual Economics and Data Science, Chinese Academy of Sciences.

The main tasks of the future intelligent laboratory include: establishing an AI intelligent system IQ evaluation system, conducting world artificial intelligence IQ evaluation; developing an Internet (city) cloud brain research plan, building an Internet (city) cloud brain technology and enterprise map, to enhance the enterprise, Industry and city intelligence level services.