1. Crises of the richness
Hundreds of components ... transforming, visualizing ...
Visual “knowledge flow” to link components, or script languages (XML) to define complex experiments.
Yale 3.3: type # components
Data preprocessing
Experiment operations
Learning methods 114
Metaoptimization schemes
Postprocessing
Performance validation
Visualization, presentation, plugins ...
Are new methods better than what we already have in our treasure box? How can we be sure?

2. Meta-learning as search in model space
k-NN 67.5/76.6%
+d(x,y); Canberra 89.9/90.7 %
+ si=(0,0,1,0,1,1);
71.6/64.4 %
+selection,
67.5/76.6 %
+k opt; 67.5/76.6 %
+d(x,y) + si=(1,0,1,0.6,0.9,1); Canberra 74.6/72.9 %
+d(x,y) + selection; Canberra 89.9/90.7 %
k-NN 67.5/76.6%
+d(x,y); Canberra 89.9/90.7 %
+ si=(0,0,1,0,1,1);
71.6/64.4 %
+selection,
67.5/76.6 %
+k opt; 67.5/76.6 %
+d(x,y) + si=(1,0,1,0.6,0.9,1); Canberra 74.6/72.9 %
+d(x,y) + sel. or opt k; Canberra 89.9/90.7 %
Search in a well-defined transformation framework, from the simplest kNN to novel combination of procedures & parameterizations.

3. Difficult case: complex logic
For n bits there are 2n nodes; in extreme cases such as parity all neighbors are from the wrong class, so localized networks will fail.
Achieving linear separability without special architecture may be impossible.
Redefining goal of learning: k-separability.

4. Web/text/ databases interface
DREAM modules
Natural input modules
Cognitive functions
Affective functions
Web/text/ databases interface
Behavior
control
Control of devices
Talking
head
Text to speech
NLP
functions
Specialized
agents
Natural perception requires many specialized transformations, not genera learning techniques; cognitive functions go beyond pattern recognition, to learning from partial observations and systematic reasoning.

5. Cognitive Systems, ICANN panel, Q1
What is machine intelligence, as beyond pattern matching, classification and prediction.
Low level cognitive functions: perception, sensorimotor actions, are basically active signal analysis (control used to get better signal) + active pattern matching (anticipation, attention, information filtering) to recognize objects and structures.
Higher-level cognitive functions: associative and episodic memory for natural perception, representation of complex knowledge structures, sequential logical and intuitive reasoning processes, problems solving, planning and other things symbolic AI works on ...
In between? Reinforcement learning, emotions?
Intuitive computing, solving compositionality problems – search constrained by separable neural networks.

6. Cognitive Systems, ICANN panel, Q2
How can such machine intelligence best be employed?
There are already numerous educational + industrial applications, more are coming in home and office automation, cars (vision and object recognition, planning routes) etc. Driving in urban environment requires some pre-symbolic reasoning.
We need a detailed roadmap with progressively more difficult tasks:
what has been already done and may be integrated in other models to avoid duplication of work (although sometimes it is useful), may be used in applications & improved; sound/object localization, orientation mechanisms, control, recognitions of speech, gestures, lip movements, face recognition, person identification, etc;
what is doable in relatively short time – some emotions, object recognition, attention control;
what is difficult – neural approach to higher mental functions?

7. Cognitive Systems, ICANN panel, Q3/4
How is intelligence actually achieved in the human brain (for example as related to recent researches on the capacity and power of human working memory)?
Depending on the level.
Perception, motor control – good models of some functions.
Higher cognitive functions - no one really knows?
How is reasoning achieved without language?
General idea: at the base level, spreading activation networks, particular configuration of activation distributions represents the object at microlevel;
different hierarchical levels of search, left/right hemisphere interactions – interesting experimental data from paired word associations and solving problems requiring insight.
General principle: learning new by re-using old.

8. Cognitive Systems, ICANN panel, Q5/6
What are general simple architectures that support reasoning?
Classical symbolic: SOAR, ACT-R, have large number of applications, although they are very rough approximations to brain processes.
Interesting connectionist architectures: IDA (Franklin), Shruti (Shastri) and many others.
Comparison of some architectures in real-time robot control applications would be useful.
How can we implement primitive levels of reasoning as are observed in crows and chimpanzees?
Animal reasoning is pre-symbolic, so first sensorimotor exploration is needed, involving object and motion recognition + solving simple manipulation problems.

9. Cognitive Systems, ICANN panel, Q7/8
Does language play an essential role in the reasoning process (sometimes hidden)?
Representation of real objects and sensomotoric sequences in terms of activations has large variability, adding symbolic labels reduces variability in the part of activation space. This must influence the reasoning process.
How can we build a truly creative architecture to solve difficult tasks?
I’ve proposed (WCCI’06) to focus first on creation of new words, starting from description of products, organizations etc, simulating the process, as our simulations find some interesting words and about 2/3 words that have already been invented.
This can be extended to higher-level mechanisms, as in Mazursky, Goldberg and Solomon work on ideas for advertisement.

10. Cognitive Systems, ICANN panel, Q9/10
How would a reasoning system at non-linguistic level help in any branch of industry?
One example is in understanding complex machinery reactions, as in the refineries or other plants; this is relatively simple and may be achieved using correlation machines.
Car driving in urban environments will need some reasoning.
What are the ethical problems thrown up by future advances in this area, advancing as it does towards the 'soul' of humanity?
People are very resistant to science and will harbor their ideas about souls and spirits independent of the development ...
Problems may arise in distant future when more and more jobs will be automated.
Conscious machines will open a Pandora’s box ...