PICT: patterns change locally and through neighbors. â«. IDR: locality over overlaid sheets. â«. Other problems: Extracts spatial propositions (humans do it by.
What Does It Mean for a Computer to do Diagrammatic Reasoning? B. Chandrasekaran
B. Chandrasekaran 2002
What is Diagrammatic Reasoning? For these problems, consider the diagram drawn on paper vs imagining vs a computer program doing it.
Consider some examples: 1. Proving syllogisms using Euler Diagrams:
1 + 3+ 5 +…(2n-1) = n 2
(i) Concrete version vs the infinite version (ii) Subtle interplay of what perception gives and how it is combined with conceptual information n
Examples, contd PICT (Furnas) Humans not especially good in doing this by imagining these.
Examples, contd A is left of B, B is left of C and C is left of D
A
B
C
D
A is poorer than B, B is poorer than C and C is poorer than D
Imagine taking a step forward, a few steps to the right, and a step back. Where are you with respect to the starting point? Anderson’s IDR Interdiagrammatic E.g., Produce a new diagram from two diagrams such that each pixel is a function of the intensity on each of them
Examples, contd Reasoning about maneuvers n
n
Movement’s are diagrammatically represented over real terrain map. The movement notations contain iconic as well as spatial information in interesting ways.
What is common between these? They all use operations on data structures that mirror some characteristic structure of 2-D space. n
Mathematically, 2-D space is a set of points characterized by a metric in which a point “connects” to another point through other points that are shorter in distance; w interacts with another point by propagating interactions through
neighboring points. Locality.
n n n
n
PICT: patterns change locally and through neighbors IDR: locality over overlaid sheets Other problems: Extracts spatial propositions (humans do it by visual perception). Some of the operations are easy for human perception and imagination, some not so easy, but humans can use external props in some cases.
Use of diagrams in problem solving For simplicity, I’ll focus on use of diagrams in goaldirected problem solving and reasoning. n
The system moves through a series of cognitive states, each of which is a proposition inferred from other propositions, or obtained by perception on the diagram (or more generally external representations). Conceptual Memory
Diagram
Perception Propositions from Conceptual Memory Propositions from Perception Problem Solver,Goals
Action
Three noteworthy points about use of diagrams 1. As mentioned, diagrams make some relevant information available for pickup by perception. 2. Diagrams as abstractions: they combine iconicity and spatiality in interesting ways. n
Icons (labels, symbols, etc.) serve as pointers to conceptual knowledge. w E.g. railroads shown as hatched lines, the axiality serving
a spatial purpose, but the hatching serving an iconic role.
3. Diagrams as model instances.
DR as a type of model instancebased reasoning A is left of B, B is left of C and C is left of D. A
B
C
D
The diagram here is a model instance
A General Type of Proof: A general proposition is stated. The reasoner constructs a model instance, finds some way to evaluate the proposition in the model, then using using some “warrant for generalization,” declares the general proposition. Goes beyond DR.
Requirements for use of model instance-based proofs. 1. We need to have a way of evaluating the predicate of interest in the model instance. n n
This is not always guaranteed. Human perception can evaluate certain spatial propositions for certain spatial models.
2. Warrant for generalization is generally hard to come by. n
In general, you can’t generalize from an instance.
When DR (or MIBR in general) works in a certain situation, it is by virtue of the fact conditions 1 and 2 are satisfied. When people make mistakes, we can locate the error in 2.
Then how come DR is so common in commonsense and professional reasoning? Knowledge of diagrammatic mappings with properties 1 and 2 is hard-earned human knowledge. n
For classes of problems, our culture, our educational system teaches us families of mappings. w Invention of such mappings is a prized event in the progress of human knowledge: quantities, temporal phenomena as linearly mappings, maps and abstractions, are breakthroughs. n
Still people do make mistakes.
w In professional disciplines, invention of such representations is a major event as well, and inventors are honored: Names such as Feynman, Euler, Venn, etc., resound through the ages.
Real diagrams versus imagining How does perception come in? There is no perceptual system inside the head. n
n
Pylyshyn’s objections to standard accounts of image use in problem solving. The issue is the degree to which the structure of space is simulated when imagining.
Similarly, unless we have in mind a robot with a visual system looking at a diagram on paper, not clear what it means for a computer program to be doing diagrammatic reasoning? n
Can we seek a functional level at which we can make a claim about Diagrammatic Reasoning?
A non-diagrammatic proof Consider a non-diagrammatic way to solve: A is poorer than B, B is poorer than C and C is poorer than D. Axioms:
(i) P(A,B), (ii) P(B,C), (iii) P(C,D)
∀x, y P(x,y) → R(y,x) (1)
Prove: P(A,D)
∀x,y,z [P(x,y) ∧ P(y,z) → P(x,z)](2) ∀ x ~P(x,x) (3) ∀x ~R(x,x) (4)
From (i), (ii) and (2), P(A,C)
(iv)
From (iv), (iii) and (2), P(A,D)
Compare structure of this proof with DR for the same problem.
Compare two structures (Universally quantified) premisses
(Universally quantified) premisses
Model instance Spatial manipulation of model and perceptual extraction of instance-specific conclusion.
General inference rules
(Universally quantified) conclusions
These two structures are not just notational variants of each other
(Universally quantified) conclusions
Generalization
DR as a Functional Notion If a problem solving activity can be characterized in a consistent way as using one or more of the following for steps in generating a new problem state: n
Propositions extracted by perception on the concrete diagrammatic instance w E.g., this forms a square with a side of 3 units, C is to the right of A
n
New states generated by scanning, or translation, rotation, motions,or shape changes of objects, by overlays of spatial configurations. w Essentially, the generator uses a model of space
n
New states generated by rules that are triggered by conditions recognized by perception.
Then we say that that cognitive state change was functionally diagrammatic. n
But this is a relative notion
Functionality: Level 1 Mathematical structure of space, 2-D grid space: obeys a metric n n
Neighbors are “closer.” Consider internal data structure whose link structure mimics the metric of the 2-d space (like a cellular automaton). w Such a DS can support many of the basic operations on
space, such as scanning, grouping. In principle, this DS supports scanning, motion/translation/rotation, overlay, and without loss of generality (modulo the approximation of space by the grid structure) supports all percept formation operations, including perceptual grouping. n
n
Pylyshyn argues against the existence of such operations in the mind. New states are generated from prior knowledge. No such limitation for computers, except time complexity.
Functionality: Perceptual grouping into objects Perception produces a perceptual experience as well as propositions containing object recognition statements and relational statements. Perceptual experience involves organizing the marks on the paper into perceptually meaningful groups: points, lines, regions. This is prior to recognition and naming. Propositions are typically open-ended, and the grouping can be openended in complex cases.
n
n
7 groups recognized: 3 points (A,B,C), 3 lines (AB, BC, CA), and a region (ABC). Relations between groups. A flexible DS is needed to perceive “as” in different ways. Open issues in the ontology of points, lines and regions,
A
B
C
perimeters, vertices, etc. Not all organizations are immediately perceived for the
A
figure on the left.
C D
E
B
Alternative organizations, involving interpretation shift
Data Structure combining the levels Level 2 supports object naming and relational propositions n
n
Perceptual grouping can be a DS layer above the Level 1 layer, with elements of objects linked to the elements in Level 1. Level 2 can be the only level, i
Functionality: Level 2 Just relational predicates are extracted from specific spatial specifications of objects, but scanning, movement, etc., don’t mirror the fine structure of space. n n
E.g., moving one step forward. A computer program that uses the MIBR proof structure by computing relational predicates, but not by local operations on spatial information, is functionally diagrammatic at this level, but is not at level 1 (similar to human imagination, a la Pylyshyn).
Summary DR was characterized in terms of DS’s that mirror the structure of space at different levels. Diagrammatic problem solving/reasoning was characterized as a species of a type of proof, ModelInstance-Based proof, which requires a number of stringent conditions to work. n
Amazingly, classes of mappings for classes of problems exist and are a prized possession of human cultural and professional knowledge.
DR by imagination and by computer programs share several properties in common, and can be diagrammatic at different functional levels, but that they can do DR is not incoherent.
DR is a precursor to a new way of thinking about the architecture of cognition. Traditional CogSci/AI has been dominated by a narrow symbolic language of thought view. n
Turing symbols, abstract thought, …
But perception is prior to high level thought, and many ideas are emerging in which the basic architecture of cognition is at least as perceptual/kinesthetic as it is conceptual. n
Brookes Cog, Barsalou’s Perceptual Symbol Systems, my own multi-modal cognitive architectures
In that sense, DR is not simply a “technique,” but it touches on deep new issues in cognition.
