I have recently came across an AGI conference 2015 paper [Schmid2015] comparing various programs solving number series problems. A detailed investigation was held on IGOR2 program and Artificial Neural Networks (ANN) approach. As series of 20 numerical tests were conducted. I reran these tests on IQSolver and it scored better (twice less failures).
Test details
IGOR2 was given 5 elements in series as input - it succeeded in 14 series and failed in 6. When IQSolver was also given 5 elements - it succeeded in 17 and failed in 3. By failure I consider giving different results than seems most natural for humans.
According to [Schmid2015] ANN was given 7 elements and had to predict the 8th. However, some series have only 7 elements. I assumed that in such case 6 elements were given as input. I performed the same test on IQSolver. The result for ANN was 3 failures, whereas IQSolver failed in just 1 case. Unlike IGOR2 and ANN, IQSolver was not adjusted in any way to this test - I used version available online from August 2015.
Here are the results for IQSolver. It does not produce formula but if you click on each series you will see a generated explanation along with continuation of the series. For the full series - last number is not send to IQSolver. You may first read description of how IQSolver works. I have not yet written formal, mathematical description. Do you think it would be useful?
Discussion about wrong results
S15 - 6,9,18,21,42,45,90,93
The correct formula is if (even; f(n-1) + 3; f(n-1) * 2), which means you should alternately increase the previous number by 3 and multiply by 2. If all 8 numbers are given then IQSolver finds the correct answer (the generated numbers match the rule) although it explains it in a more complicated way (deduced numbers are in italic):| Solution #1 | [6, 9, 18, 21, 42, 45, 90, 93, 186, 189, 378, 381, 762, 765, 1530, 1533] |
| Explanation #1 | [6, 9, 18, 21, 42, 45, 90, 93, 186, 189, 378, 381, 762, 765, 1530, 1533] may be divided into 2 sequences:
[6, 18, 42, 90, 186, 378, 762, 1530]; [9, 21, 45, 93, 189, 381, 765, 1533]
Apply "-" on consecutive elements in [6, 18, 42, 90, 186, 378, 762, 1530] to get [12, 24, 48, 96, 192, 384, 768]
Apply "/" on consecutive elements in [12, 24, 48, 96, 192, 384, 768] to get 2
Apply "-" on consecutive elements in [9, 21, 45, 93, 189, 381, 765, 1533] to get [12, 24, 48, 96, 192, 384, 768]
Apply "/" on consecutive elements in [12, 24, 48, 96, 192, 384, 768] to get 2
|
| Complexity #1 | 5.5 |
It divides the series into 2 subseries. First subseries 6, 18, 42, 90 follows the formula (f(n-1)+3)*2 = 2*f(n-1) + 6. The second 9, 21, 45, 93 is just (f(n-1)*2)+3 = 2*f(n-1)+3 . These 2 linear functions are easily discovered. However, IQSolver treats both subseries independently and does not check that they have the same transformation and thus the complexity should be smaller.
Moreover, if one less input number is given - it does not find enough justification to assume 12, 24 is a series with constant quotient. To assume a certain arithmetic operation between consecutive elements always return constant value at least two values must be returned. That is why 12, 24, 48 is enough to induce the rule.
Therefore, for 7 input numbers another answer is returned:
| Solution #1 | [6, 9, 18, 21, 42, 45, 90, 81, 174, 117, 318, 117, 582, -15, 1122] |
| Explanation #1 | Apply "-" on consecutive elements in [6, 9, 18, 21, 42, 45, 90, 81, 174, 117, 318, 117, 582, -15, 1122] to get [3, 9, 3, 21, 3, 45, -9, 93, -57, 201, -201, 465, -597, 1137]
Apply "-" on consecutive elements in [3, 9, 3, 21, 3, 45, -9, 93, -57, 201, -201, 465, -597, 1137] to get [6, -6, 18, -18, 42, -54, 102, -150, 258, -402, 666, -1062, 1734]
Apply "-" on consecutive elements in [6, -6, 18, -18, 42, -54, 102, -150, 258, -402, 666, -1062, 1734] to get [-12, 24, -36, 60, -96, 156, -252, 408, -660, 1068, -1728, 2796]
Obtain element by applying "-" on previous 2 elements of [-12, 24, -36, 60, -96, 156, -252, 408, -660, 1068, -1728, 2796]
|
| Complexity #1 | 5.0 |
And for 5 input numbers the answer is different:
| Solution #1 | [6, 9, 18, 21, 42, 57, 102, 153, 258, 405, 666, 1065, 1734] |
| Explanation #1 | Apply "-" on consecutive elements in [6, 9, 18, 21, 42, 57, 102, 153, 258, 405, 666, 1065, 1734] to get [3, 9, 3, 21, 15, 45, 51, 105, 147, 261, 399, 669]
Apply "+" on consecutive elements in [3, 9, 3, 21, 15, 45, 51, 105, 147, 261, 399, 669] to get [12, 12, 24, 36, 60, 96, 156, 252, 408, 660, 1068]
Obtain element by applying "+" on previous 2 elements of [12, 12, 24, 36, 60, 96, 156, 252, 408, 660, 1068]
|
| Complexity #1 | 6.0 |
S12 5,6,7,8,10,11,14,15
When given 7 number IQSolver finds the expected solution:Apply "-" on consecutive elements in [5, 6, 7, 8, 10, 11, 14, 15, 19, 20, 25, 26, 32, 33, 40] to get [1, 1, 1, 2, 1, 3, 1, 4, 1, 5, 1, 6, 1, 7]
[1, 1, 1, 2, 1, 3, 1, 4, 1, 5, 1, 6, 1, 7] may be divided into 2 sequences:
[1, 1, 1, 1, 1, 1, 1]; [1, 2, 3, 4, 5, 6, 7]
[1, 1, 1, 1, 1, 1, 1] consists only of : 1
Apply "-" on consecutive elements in [1, 2, 3, 4, 5, 6, 7] to get 1
It generalizes sequence of difference 1, 1, 1, 2, 1, 3 into 1, 1, 1, 2, 1, 3, 1, 4, 1, 5, 6, 1, 7.
But when only five numbers are given - sequence of differences is 1, 1, 1, 2 - there are too few repetitions to generalize. IQSolver does not generalize 1, 2 to 1,2,3,4 (althought it does for 1,2,3), maybe this should be corrected. Anyway, the solution actually found is pretty strange for humans.
Apply "-" on consecutive elements in [5, 6, 7, 8, 10, 16, 46, 286, 3406, 68926, 2296606, 124819006, 11029312606] to get [1, 1, 1, 2, 6, 30, 240, 3120, 65520, 2227680, 122522400, 10904493600]
Apply "/" on consecutive elements in [1, 1, 1, 2, 6, 30, 240, 3120, 65520, 2227680, 122522400, 10904493600] to get [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
Obtain element by applying "+" on previous 2 elements of [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
S17 8,10,14,18,26,34,50,66
Again, IQSolver finds results for 7 input numbers:Apply "-" on consecutive elements in [8, 10, 14, 18, 26, 34, 50, 66, 98, 130, 194, 258, 386, 514, 770] to get [2, 4, 4, 8, 8, 16, 16, 32, 32, 64, 64, 128, 128, 256]
Apply "/" on consecutive elements in [2, 4, 4, 8, 8, 16, 16, 32, 32, 64, 64, 128, 128, 256] to get [2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2]
[2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2] is repeated sequence of : [2, 1]
But for 5 input numbers it cannot generalize sequence of differences 2, 4, 4, 8 into 2, 4, 4, 8, 8, 16, 16, 32, 32. It is too short. The answer is "inhuman" again.
Apply "-" on consecutive elements in [8, 10, 14, 18, 26, 30, 46, 48, 112, 1.1e+02, 2.2e+03, 2.2e+03, 1.7e+07] to get [2, 4, 4, 8, 4, 16, 2, 64, 0.25, 2048, 0.00098, 16777216]
Apply "/" on consecutive elements in [2, 4, 4, 8, 4, 16, 2, 64, 0.25, 2048, 0.00098, 16777216] to get [2, 1, 2, 0.50, 4, 0.13, 32, 0.0039, 8192, 4.8e-07, 17179869184]
Obtain element by applying "/" on previous 2 elements of [2, 1, 2, 0.50, 4, 0.13, 32, 0.0039, 8192, 4.8e-07, 17179869184]
Other solvers
[Schmid 2015] mentioned other programs capable of solving IQ tests . Most notable examples are SeqSolver [Strannegard 2013] and ASolver [Strannegard 2012], which score IQ tests above 140. I do not have an executable version of these programs to test them, but from their description I infer they have less abilities than IQSolver. However, their cognitive abilities are deliberately bounded to meet human constraints.
Future development
Some improvements could be done to make IQSolver score better in IQ tests. However, solving tests is just the first step on this roadmap. The next milestones are learning rules of board games and text computer games. But, in order to make an AI capable of finding rules in complex but limited set of data a new program must be created. Therefore, I plan to build a completely new AI. Some design plans of it are given here and here.
References
[Schmid 2015] "Comparing Computer Models Solving Number Series Problems" by Ute Schmid and Marco Ragni. 8th International Conference, AGI 2015http://agi-conf.org/2015/wp-content/uploads/2015/07/agi15_ schmid.pdf
[Strannegard 2013] "Bounded Kolmogorov Complexity Based on Cognitive Models" Claes Strannegård, Abdul Rahim Nizamani, Anders Sjöberg, and Fredrik Engström
http://gup.ub.gu.se/records/fulltext/178586/178586.pdf
[Strannegard 2012] "An anthropomorphic method for number sequence problems" Claes Strannegård · Mehrdad Amirghasemi · Simon Ulfsbäcker http://engstrom.morot.org/material/bounded_kolmogorov.pdf
IQSolver is really the best :). I tried step function once and it didn't solve It, but most of "my" problems it solved well
ReplyDelete