Spatial ability learning through educational robotics

Noname manuscript No. (will be inserted by the editor)

Spatial ability learning through educational robotics ` Carme Juli` a · Oscar Antoli

Received: date / Accepted: 1 April 2015

Abstract Several authors insist on the importance of students’ acquisition of spatial abilities and visualization in order to have academic success in areas such as science, technology or engineering. This paper proposes to discuss and analyse the use of educational robotics to develop spatial abilities in 12 year old students. First of all, a course to introduce robotics to 6th grade primary school students was designed. The key intention was to prepare practical and motivating sessions in order to foster the students’ involvement in hands-on learning. Hence, during the sessions of the course, challenges were provided for the students, in order to develop their capabilities as proficient problem solvers. The teacher assisted and guided the students, and the students were encouraged to solve the problems by themselves, working in 3-members teams. The main goal of this paper is to discuss and analyse the potential usefulness of educational robotics to develop spatial abilities. To carry out the analysis, students were randomly divided into an experimental group (EG), which participated in the robotics course, and a control group (CG), which did not take part in the robotics course. The extensive existing literature for spatial ability evaluation was analysed and reviewed and a pre-test and a post-test were prepared for use in the research study. Initially, the spatial ability of both EG and CG students was assessed with the pre-test. Then, after finishing the robotics course, the same sets of students were tested with the post-test. An extensive analysis of the results is provided in the paper. Results show that the positive change in spatial ability of the participants in the robotics course (EG) was greater than change evident in the students who did not join the course (CG). The improvement was statistically significant. The results also show that the overall performance of the students depends on the instruments used to evaluate their spatial abilities. Hence, this study manifests clearly the importance of the selection of those instruments. Department of Computer Science and Mathematics, Universitat Rovira i Virgili (URV), Av. Pa¨ısos Catalans, 26, 43007 Tarragona, Spain E-mail: [email protected]

` Carme Juli` a, Oscar Antoli

2

Keywords Robotics · Spatial ability · Visualization

1 Introduction Spatial ability is defined in Sutton and Williams (2007) as the performance on tasks that require: – mental rotation of objects, – the ability to describe and understand how objects appear at different angles, – an understanding of how objects relate to each other in space. Sutton et al (2005) mention that a substantive feature of spatial abilities is three-dimensional (3D) understanding, which is the capability to extract information about 3D properties from two-dimensional (2D) representations. This skill requires perceptual abilities to interpret what is seen, and spatial abilities to mentally manipulate graphical representations (Sutton and Williams (2007)). In NCT (2000), the National Council of Teachers of Mathematics (NCTM) released a document entitled Principles and Standards for school mathematics. The authors set goals and recommendations for mathematics education in the prekindergarten through to grade 12. Focussing on the Geometry Standard, authors in NCT (2000) point out that geometry has long been regarded as the place in high school where students learn to prove geometric theorems, such as the Pythagorean Theorem, theorems about lines and angles, or theorems about parallelograms1 . The Geometry Standard they propose takes a broader view of the power of geometry by calling on students to analyse characteristics of geometric shapes and construct mathematical arguments about the geometric relationships, as well as to use visualization, spatial reasoning, and geometric modeling to solve problems. In addition to the NCTM’s recommendation to introduce visualization and spatial abilities at the primary school level, several authors insist on the importance of acquiring these abilities in order to have academic success in areas such as science, technology or engineering (e.g., Sorby (2009); Sutton and Williams (2007); Verner (2004)). Sorby (2009) provides a detailed report about the presence of visualization learning in engineering education and specifically presents some strategies effective in developing 3D spatial skills and in contributing to student success. Contero et al (2006), on the other hand, show that sketchbased applications, such as computer-aided generation of 3D models from 2D freehand sketching, can provide an effective way of improving spatial abilities and capturing students attention. The authors also point out how those types of applications can stimulate students’ learning thus creating a positive attitude to the sketching tasks. 1

http://www.corestandards.org/Math/

Spatial ability learning through educational robotics

3

The intention of this project was to use educational robotics for developing spatial abilities in 12 year old students. It should be noticed that the aforementioned spatial abilities are required when constructing a model of a robot. For example, students need to be able to visualize mental rotations and relative positions of the components that form the robot.

1.1 Additional background During the 1970s, Papert (1980) proposed the use of new technologies, such as computers and robots, to change the nature of learning at school. He developed the first software designed exclusively for use by children, the Logo programming language, and demonstrated that children of almost any age could learn to program in Logo under good learning conditions with plenty of time and the use of powerful research computers. The Logo programming language consists of a small mobile robot, a turtle, which moves in response to programmed commands that are relative to its own position. When children execute the program, they can analyse if it has worked as expected. Since the seminal proposal (Papert (1980)), there has been a considerable increase in the research literature about the use of robotics as an educational tool (e.g., Barak and Zadok (2009); Barker and Ansorge (2007); Benitti (2012); Datteri et al (2013); Highfield (2010); Rogers and Portsmore (2004)). Some of these research studies are analysed in Section 2. Most of these existing studies point out the multidisciplinary nature of robotics and how it can improve and motivate students’ capabilities such as problem solving, creativity, and working in groups. Moreover, educational robotics presents a wonderful opportunity to introduce children to the world of technology (Bers and Portsmore (2005)). While it is true that the students are motivated when working with robots, it should be remarked that their motivation depends also on the way the embedded concepts and processes are taught. However, Rocard et al (2007) mention that, in recent years, many studies have highlighted an alarming decline in young people’s interest for key science studies and mathematics. They also mention that the origins of that declining interest are largely found in the way science is taught at school. Rocard et al. point out that there is a need to prepare young people for a future that will require a deep understanding of scientific knowledge and an understanding of technology. They propose to use inquiry-based science education methods (IBSE), since those methods emphasize features such as curiosity and observations followed by problem solving and experimentation. Educational robotics can provide opportunities for motivating students, who gain a sense of power over technology when the machine (robot) does what they order (Barker and Ansorge (2007)). Papert (1980) claims that people learn better when they are engaged in designing and building their own personally meaningful artifacts. This is the basis idea of the constructionist philosophy of education (Papert (1980)). Based on this idea, existing research


4

(e.g., Barak and Zadok (2009)) can be underpinned through the use of robotics project-based learning at schools. Hence, this study involves the use of a robotics course which is founded on inquiry-based science education methods (IBSE), as well as the concept of the constructionist philosophy and the use of robotics projects, in order to motivate the students. Educational robotics allows the adoption of hands-on learning, whereby students can be readily involved in the classroom activity work. The idea is to propose challenges to the students, who have to solve those challenges by working in groups.

1.2 Objective The main goal of the current paper is to analyse the potential usefulness of educational robotics in developing spatial abilities in 12 year old students. In order to carry out the analysis, the students are randomly divided into an experimental group (EG) that participates in the robotics course and a control group (CG) that does not take part in the robotics course. In order to evaluate the initial spatial abilities of both EG and CG students, a pre-test is administered. After the completion of the robotics course, both groups are tested again, through a post-test. The working hypothesis is that the students who engage in the robotics course (EG students) have an improved spatial ability compared to those students who do not engage in the robotics course (CG students). Any improvement is to be tested for statistical significance. The remainder of the paper is organized as follows. Section 2 introduces and reviews existing experiences about using educational robotics in school. In Section 3, the proposed study is described in detail. The results obtained by various individuals and groups are presented, compared and analysed in Section 4. The final conclusions and intentions for future research are examined and discussed in Section 5.

2 Robotics at school This Section reports a review of existing approaches that include the use of educational robots to introduce concepts related to mathematics, science, engineering or technology. As far as the authors are concerned, the most used educational robots are LEGO. In fact, the FIRST LEGO League (FLL) is one of the most popular robot competitions. FLL releases a Challenge, which is based on a real-world scientific topic. Over 265.000 children aged between 10 and 16, from over 80 countries, participated in the last edition of the Challenge, the FLL 20142 . Rogers and Portsmore (2004) pointed out that incorporating engineering in the elementary school curriculum provides students with ways of connecting, applying, and reinforcing knowledge in mathematics, science, and design. The 2

http://www.firstlegoleague.org/challenge/2014fllworldclass


5

authors develop an effective platform to teach engineering and their goal is to excite students about engineering, mathematics, and science, and to teach these disciplines in a hands-on and practical way. Thus, they can introduce STEM education (science, technology, engineering and mathematics) through practical activities that (1) can stimulate and improve the interest of the students in those areas and (2) can connect the lessons with the real world. The authors describe educational experiences they have conducted for over 15 years, by using robots (LEGO) and the ROBOLAB software. Indeed, part of their work helped establish the new engineering standard in K-12 education for the state of Massachusetts (Eng (2006)). Barker and Ansorge (2007) mention that the advances in technology have brought down the cost of robots and made it easier to implement their use into classrooms. The authors provide results showing that the use of a science and technology curriculum based on robotics increased the achievement scores in science, engineering and technology of the participants in an after school program. Barak and Zadok (2009) have also proposed robotics projects as a way of introducing concepts in science, technology and problem solving. They described a study that takes place within a framework of a robotics course offered to junior high school pupils. They concluded that implementing informal instruction into a project-based program was a high quality approach compared to engaging with procedural knowledge learned by rote. More recently, Rockland et al (2010) explored best practices for bringing engineering into the science and mathematics curriculum of secondary school classrooms by describing a project that utilized concepts representing the merger of medicine, robotics, and information technology. The authors state that the design, construction, and control of the robots by the students contributed to the learners’ acquisition of relevant knowledge (such as physical and chemical concepts) and the refinement of their thinking skills related to science, engineering design, and information technology. Highfield (2010) described a series of tasks using Bee-bots and Pro-bots, developed as part of a larger project examining three and four year old children’s use of robotic toys as tools in problem solving. The key idea was that children program the robotic toys and observe their various movements. Through the proposed activities, various mathematical concepts and processes were promoted: spatial concepts (e.g., capacity, angle of rotation, directionality position on a plane), measurement, numeracy, problem solving (e.g., estimation, evaluating solutions, trial and error) and representation. Verner (2004) used a learning environment (RoboCell) where manipulations of the objects were performed by robot operations that required spatial thinking. The author proposed a curriculum related to robot kinematics and point-to-point motion, rotation of objects and robotic assembly of spatial puzzles. Pre and post-course tests performed in middle and high schools demonstrated the improvement of the spatial abilities of students in tasks that were practised in the course.


6

Further, Coxon (2012) has presented results showing that a treatment with LEGO robotics produces significant and meaningful mean gain scores on a measure of spatial ability for 9 to 14 year old students. The author asserts that spatial ability is important to STEM success in higher education, career success, and innovations. These capabilities are important to our quality of life and economic improvements, hence, it is clearly evident that LEGO robotics should be incorporated into practice at school. The above research studies are in accord regarding the importance of developing students’ spatial abilities and also the usefulness of introducing educational robots at school as a way of enhancing students’ capabilities in this area.

3 Proposal This Section presents in detail the proposed study, which focused on quantifying the acquisition of spatial abilities by students as they engaged in an educational robotics program at primary school.

3.1 Objective The main objective of this work was to analyse the potential of educational robotics in developing spatial abilities in 12 year old students. In order to achieve that objective, a course was designed to introduce robotics to 6th grade primary school students. The idea was to prepare practical sessions, in which the students tackled challenges proposed by the teacher. The aim was to compare the spatial ability acquired by the participants in the robotics course compared to the spatial ability acquired by the students who did not participate in the course. To carry out the comparison, students were randomly divided into an experimental group (EG), which participated in the robotics course, and a control group (CG), which did not take part in the robotics course. Data collection occurred through a 1 hour session per week during 10 weeks, from February to April. The course was divided into three parts, as can be seen in the outline shown in Table 1. Details on the course are provided in Section 3.6.

3.2 Material The materials used in the sessions consisted of three different Fischertechnik3 sets containing more than 1000 components (see Fig. 1 for illustrations of sample sets). The Universal 3 was intended as a starter for model construction; 3

http://www.fischertechnik.de/en/home.aspx


7

Table 1 Outline of the course session

date

part

content

1

7th February

2 3

14th February 21st February

1

introduction to robotics and model’s construction

4 5 6

28th February 7th March 14th March

2

model’s construction and introduction to programming

7 8 9

21st March 28th March 4th April

3

introduction to computer-aided design (CAD)

10

11th April

post-test

(a)

(b)

pre-test

(c)

Fig. 1 Kits used in the robotics course: (a) Universal 3 (> 500 components); (b) ROBO LT Beginner Lab (> 200 components); (c) Oeco Tech (> 320 components)

the ROBO LT Beginner Lab is suitable for programming; and the Oeco Tech consists of constructions that are based on renewable energies. The kits include sensors (e.g., phototransistors, mini-switches), actuators (e.g., XS motor, indicator lights) and the ROBO LT Controller that transmits orders from a computer to a robot. The Controller has 3 inputs for sensors, 2 outputs for motors or indicator lights and a USB interface for simultaneous power supply. The ROBO Pro Light software is used to program the model robots. A snapshot of the software is shown in Fig. 2. The Fischertechnikdesigner software is used for the computer-aided design (CAD). This software allows the construction of realistic models of the robots, since it has all the components of Fischertechnik in 3D (see a snapshot of the workspace in Fig. 3).

3.3 Context The study described in this paper was conducted in a primary school in a small city, Asotrot, Spain. The study was carried out with a 6th grade class. There were 21 12 year old students (11 boys and 10 girls) in the classroom. Due to the fact that only 3 sets of kits were available in the school, and having in mind that the ideal number of students per set is 3 (deduced from

8


Fig. 2 Snapshot of the ROBO Pro Light software

Fig. 3 Snapshot of the Fischertechnik-designer software

previously experiences with these sets), only 9 students participated in the robotics course. These students were randomly selected, as explained below. The robotics course was presented on Fridays, at the scheduled Maths Workshop time. Hence, students who did not go to the robotics course went to the Maths Workshop. In general, the Maths Workshop time was used to promote problem solving skills in students. It should be highlighted that the current study does not form part of the annual school planning. It was a pilot project proposed by the authors. The school kindly offered 10 sessions from the Maths Workshop for conducting the key aspect of the research project. Another key consideration of the project was that the classroom environment should be a large space and there must be computers present. Therefore, the chosen environment was the school’s library. There were big rectangular


9

tables that were suitable for working in groups. Furthermore, the space was very well illuminated, which is crucial when working with such a large amount of different components. Additionally, there were some computers required for the programming activities and for the 3D design requirements.

3.4 Methodology As described above, in order to analyse the potential of educational robotics, the students were randomly divided into 2 groups: the experimental group (EG) and the control group (CG). The former was composed of 9 students (4 boys and 5 girls) and they joined the robotics course presented in this paper. The control group was formed by the remaining 12 students who did not go to the robotics course. The main purpose of this study was to determine if the students who participated in the robotics sessions develop spatial abilities in a more significant way than the ones who did not participate in the robotics sessions. This evaluation can be performed by means of a test procedure (e.g., Sorby (2009)). Therefore, in the first session, all the students (CG and EG) completed a test (pre-test), as shown in Table 1. After the 8 sessions of the robotics course, a post-test was used to detect if the spatial ability of the participants of the course (EG) had been improved significantly (based on test of significance). Details on the characteristics of the tests are provided in Section 3.5. Additionally, various products (artifacts) were collected at the end of each session. These products were analysed and the resulting evidence used to evaluate the performance of the students during the sessions. Namely, the working files of all the groups were saved, along with pictures and video records of the processes and of the physical constructions engaged in by the students. In addition, the teacher made continuous observations of the students and kept notes of every interesting fact, or event, that occurred during the sessions.

3.5 Instruments There exists a wide range of literature that aims at evaluating spatial abilities through the use of tests. Some of the most popular tests according to Contero et al (2006); Metz et al (2012); Sorby (2009) are the following: Mental Rotation Test (MRT); Differential Aptitude Test-Spatial Relations (DAT-SR); Mental Cutting Test (MCT); Purdue Spatial Visualization Test (PSVT:R); and Vandenberg MRT. The problem is that all these tests are not designed for 12 year old students, but for older ones. Therefore, they were deemed not suitable for the current study. Humphreys et al (1993) proposed the use of several sub-tests instead of using a single test in order to obtain more robust results when evaluating spatial abilities. Taking this idea into account, the instrument used in this study to evaluate the spatial abilities of the students (pre- and post-test) consisted


10 Table 2 Pre- and post-test composition sub-test

name

parts

items

1 2 3

Paper Folding Test (Ekstrom et al (1976)) Card Rotations Test (Ekstrom et al (1976)) Cube Comparisons Test (Ekstrom et al (1976)) Perspective Taking Spatial Orientation Test (Hegarty and Waller (2004))

2 2 2

20 20 42

1

12

4

Fig. 4 An item of the Paper Folding Test

of 4 sub-tests. Three of the considered sub-tests were based on Bakker’s proposal (Bakker (2008)), in which the author analyses different tests for evaluating the spatial ability of 11 year old students. The other sub-test considered for use in the current study was selected from the Spatial Intelligence and Learning Center4 , which is a web site that provides sets of tests to evaluate spatial skills. Specifically, the selected sub-test was the Perspective Taking/Spatial Orientation Test (Hegarty and Waller (2004)). Table 2 summarises the number of items of each sub-test. Note that their items are split into two parts, corresponding to the pre- and post-test, respectively. A brief explanation of each sub-test and a relevant example of its contents is provided below. Paper Folding Test. The students have to imagine how to fold and unfold a sheet of paper. Instructions to fold the paper are given on the left (see Fig. 4). Then, a hole is made in the paper. Once unfolded, a single figure on the right corresponds to the original paper on the left (in the example given in Fig. 4, the correct answer is C). Card Rotation Test. This test requires mental rotations of objects. The students have to decide if the objects on the right correspond to the object on the left, in which case they mark S (same). They mark D (different), if otherwise (see the sample responses given in Fig. 5). Cube Comparison Test. This test requires mental rotations of objects in 3D. The cubes contain a different symbol in each face. The students have to decide if the two given images correspond to the same cube (see two examples in Fig. 6). Perspective Taking/Spatial Orientation Test. This test requires the visualization of different perspectives and orientations of objects in space. An example is shown in Fig. 7. There are always the same objects on the top. 4

http://www.spatiallearning.org/


11

Fig. 5 An item of the Card Rotations Test

Fig. 6 Two items of the Cube Comparisons Test

Fig. 7 An item of the Perspective Taking/Spatial Orientation Test. The instructions given in the middle are: You are standing at the flower facing the tree. Point to the cat. The dashed line on the bottom corresponds to the expected answer

Information about the position of the student is provided in the middle. On the bottom, the student has to translate the given information to a scheme. It should be highlighted that none of the previously mentioned activities is directly addressed in the robotics course.


12

3.6 Sessions Design This Section summarises the main characteristics of the sessions provided at the primary school. As mentioned above, the idea was to design practical and motivating activities, thus encouraging the adoption of hands-on learning. The role of the teacher was to guide and assist the students, but the ultimate goal was that the students should tackle the proposed problems by themselves. Session one. In this session, all the students completed the pre-test, which was structured as shown in Table 2. Only half the items of each sub-test were used to compose the pre-test. The students have 3 minutes to finish each of the first three sub-tests (sub-tests 1, 2 and 3), which have 10, 10 and 21 items, respectively, and 2 minutes and 30 seconds to finish the sub-test 4, which has only 6 items. The steps to answer each of the sub-tests were the following: – – – –

The The The The

teacher stated the name of the sub-test and explained it briefly. teacher showed an example of the problem using a projector. teacher answered doubts. students completed the sub-test.

Session two. This session consisted of an introduction to robotics. Moreover, the methodology followed in the course was also presented. The 3 teams comprised of 3 students each were created (green, red and blue teams). The intention was that the 3 teams work with all the kits during the course. The working rules were also presented to the teams, that is, the students should be aware of the importance of taking care of the materials and also of working quietly in groups. The session finished by showing the kits to the students and explaining their main characteristics, such as the sensors and actuators they include. Session three. The students manipulated the kits for the first time. Each team was assigned a particular kit together with a model’s construction related to it. Furthermore, the students had to answer several questions about their assigned constructions. The teacher guided and assessed the students. However, they had to solve the problems by themselves. Fig. 8 shows the construction assigned to the green, red and blue team, respectively. The blue and green teams had to construct their model on a baseplate that required the use of coordinates (see Fig. 9 (a)). Fig. 9 (b) and Fig. 9 (c) show the final constructions of the red and blue teams, respectively. Notice that the students have to visualize the components that form the assigned construction and mentally rotate them in order to construct the model. Therefore, spatial abilities and visualization are essential requirements for building these assigned models. Sessions four to six. The ROBO Pro Light software and the ROBO LT Controller were introduced at this part of the course. As in the previous part, each team had to solve a challenge (see the constructions assigned to each team in Fig. 10). In these activities, they had to program the operation of their


(a)

(b)

13

(c)

Fig. 8 Assigned constructions: (a) merry-go-round (green team); (b) fan (red team); (c) solar fan (blue team)

(a)

(b)

(c)

Fig. 9 (a) Green team working on the merry-go-round construction; (b) final construction of the red team (fan); (c) final construction of the blue team (solar fan)

respective constructions in order to provide instructions for the artefact to undertake various tasks. Fig. 11 shows examples of the programming process, where all the components are connected (computer, ROBO LT Controller, and construction). The blue team had to program a traffic light, whereas the red and green teams had to install an electric motor and program the robot accordingly. The overall goal was to get the cyclist and the centrifugal force regulator moving, respectively. As described previously, the teams were assigned some tasks related to their constructions. An example of a task assigned to each of the teams is provided below: – Red team: Make the cyclist execute a single turn of the big wheel. It has to stop during 2 seconds. Then, make the cyclist do 3 turns. – Blue team: Program the traffic light for the pedestrian. First, the traffic light is red. Once the button has been pressed, the traffic light must be red 3 seconds more. Then, it has to change to green and stay like that during 8 seconds. Finally, it has to change to red again. – Green team: Make the regulator turn clockwise during 4 seconds at high velocity and turn anticlockwise during 6 seconds at low velocity. Sessions seven to nine. At this part of the course, the Fischertechnikdesigner software was introduced. The aim was that the students build models in 3D by using this computer-aided design (CAD) program. They began with preliminary easy tasks that became progressively more difficult. One of the initial activities consisted of fitting two components together. Fig. 12 shows an


14

(a)

(b)

(c)

Fig. 10 Assigned constructions: (a) cyclist (red team); (b) traffic light (blue team); (c) centrifugal force regulator (green team)

(a)

(b)

Fig. 11 Programming process: (a) cyclist (red team); (b) traffic light (blue team)

(a)

(b)

Fig. 12 Fitting two components together: (a) one of the components; (b) fitted components

example of the process. Potentially, rotations and translations could be applied to the components (see Fig. 12 (b)). Another task consisted of building a free model from a set of construction pieces and reproducing it in 3D by using the Fischertechnik-designer software. Nevertheless, the most interesting challenge that the students had to tackle in this part of the course was The quality control task. The aim was to propose a realistic problem such as the industrial fabrication of an artefact, which consists of tree stages: design; fabrication; and quality control. The idea was that each team should design a free construction by using the Fischertechnikdesigner software. Then, the teams had to go to a different computer and build the construction designed by their classmates. Finally, the teams changed the computers again and checked if the construction as displayed in the computer and the physical artifact are identical (control quality).


(a)

(b)

15

(c)

Fig. 13 Final constructions: (a) red team; (b) blue team; (c) green team

The course finished with a concluding activity, in which the students had to design, build and program a model that included an electrical component (such as an electric motor). Fig. 13 shows the final constructions performed by each of the teams. Session ten. In this last session of the course, the respective CG and the EG groups completed the post-test, which was structured in a similar format as the pre-test. The process involved in answering each of the sub-tests was analogous to the process used for the pre-test.

4 Results Table 3 shows the scores obtained by each student in the respective pre- and post-tests. The score corresponds to the percentage of correct answers. The unanswered questions were computed as wrong. Specifically, the results obtained in each of the 4 sub-tests are detailed. The average score (the last two columns in the table) is obtained by computing the mean of the scores obtained in the 4 sub-tests. The students are split into CG and EG groups in the table, in order to make the comparison easier. A different representation is used in Fig. 14, where each point represents the performance of each of the students in each of the sub-tests. The x-coordinate represents the score obtained in the pre-test, while the y-coordinate represents the one obtained in the post-test. Therefore, if a student obtains a better score in the post-test than in the pre-test, the point that represents his/her overall performance will be located above the diagonal. An analogous representation is used in Fig. 15, where each point represents the average score obtained by each of the students in the pre and post-tests. Fig. 16 shows the average scores obtained in the pre- and post-tests for both the CG students and the EG students. In particular, the polygonal figures enclose data in between lower and upper quartiles (medians are represented by horizontal lines in thinner regions). In order to compare the global performance of the CG and the EG groups, Table 4 shows the means and the standard deviations (std) of the scores obtained by each group of students in each of the sub-tests. Additionally, a t-test was performed to compare the respective means obtained in the pre-


100

100

90

90

80

80

70

70 Post−test score

Post−test score

16

60 50 40 30

50 40 30

20

20

10 0

60

10

CG EG 0

20

40 60 Pre−test score

80

0

100

CG EG 0

20


100

100

90

90

80

80

70

70

60 50 40 30

60 50 40 30

20

20

10 0

100

(b)

Post−test score

Post−test score

(a)

80

10

CG EG 0

20


80

100

0

CG EG 0

20


(c)

80

(d)

Fig. 14 Individual student scores obtained in the pre- and post-tests of each sub-test: x and y-axes correspond to the pre- and post-test scores, respectively; (a) sub-test 1; (b) sub-test 2; (c) sub-test 3; (d) sub-test 4

100 90 80

Post−test score

70 60 50 40 30 20 10 0

CG EG 0

20


80

100

Fig. 15 Individual student average scores obtained in the pre- and post-tests: x and y-axes correspond to the pre- and post-test average scores, respectively.

100


17

Table 3 Individual student scores (percentage of correct answers) obtained in each of the sub-tests. The average score corresponds to the mean score of all the sub-tests. sub-test 1 pre post

sub-test 2 pre post

sub-test 3 pre post

sub-test 4 pre post

average pre post

CG

1 2 3 4 5 6 7 8 9 10 11 12

40 70 50 40 70 30 30 20 60 30 40 60

40 20 50 40 90 60 50 40 60 60 50 50

68.7 75.0 67.5 71.2 96.2 88.7 67.5 80.0 96.2 90.0 90.0 82.5

93.7 73.7 83.7 75.0 96.2 82.5 81.2 81.2 96.2 87.5 81.2 63.7

61.90 52.38 57.14 57.14 76.19 76.19 52.38 76.19 76.19 57.14 57.14 42.85

76.2 47.6 61.9 66.7 76.2 66.7 52.4 66.7 38.1 57.1 47.6 38.1

16.7 33.3 16.7 16.7 33.3 50.0 33.3 83.3 33.3 33.3 16.7 33.3

33.3 16.7 16.7 50.0 33.3 33.3 33.3 66.7 66.7 50.0 50.0 0.0

46.8 57.7 47.8 46.3 68.9 61.2 45.8 64.9 66.4 52.6 50.9 54.7

60.8 39.5 53.1 57.9 73.9 60.6 54.2 63.6 65.2 63.7 57.2 38.0

EG

1 2 3 4 5 6 7 8 9

50 40 30 30 20 30 60 40 40

60 50 40 30 60 40 70 60 50

72.5 58.7 71.2 48.7 96.2 87.5 97.5 68.7 78.7

86.2 58.7 72.5 55.0 96.2 72.5 87.5 65.0 80.0

66.7 57.1 57.1 61.9 66.7 71.4 71.4 61.9 66.7

61.9 52.4 57.1 71.4 61.9 71.4 61.9 61.9 52.4

33.3 0.0 33.3 33.3 16.7 16.7 50.0 0.0 33.3

50.0 66.7 50.0 50.0 50.0 16.7 83.3 16.7 33.3

55.6 39.0 47.9 43.5 49.9 51.4 69.7 42.7 54.7

64.5 56.9 54.9 51.6 67.0 50.1 75.7 50.9 53.9

student

Percentage of correct answers

75 70 65 60 55 50 45

Pre−test Post−test

40 CG

EG

Fig. 16 Individual student average scores in the tests

and post-tests. Specifically, the students completed a two-tailed test, setting α = 0.05 as a significance level (5%).


18

Table 4 Pre-and post-test scores (CG and EG) (mean and std of the whole group)

sub-test

mean (%) pre post

CG std (%) pre post

1 2 3 4 average

45.0 81.1 61.9 33.3 55.3

16.8 11.0 11.5 18.8 8.3

50.8 83.0 57.9 37.5 57.3

16.8 9.6 13.3 20.2 10.2

t

p

-0.85 -0.44 0.78 -0.52 -0.51

0.40 0.66 0.44 0.60 0.61

mean (%) pre post

EG std (%) pre post

37.7 75.5 64.5 24.1 50.5

12.0 16.3 5.4 16.9 9.1

51.1 74.9 61.4 46.3 58.4

12.7 13.8 6.9 21.7 8.8

t

p

-2.28 0.92 1.08 -2.42 -1.87

0.03 0.09 0.29 0.02 0.07

4.1 Discussion The results summarised in Table 3 show that the improvement in the performance of the students is different in each of the sub-tests. Notice that in the Paper Folding Test (sub-test 1) and the Perspective Taking/Spatial Orientation Test (sub-test 4) there is a clear gain on the scores in the corresponding post-tests. In the other sub-tests (sub-tests 2 and 3), on the contrary, the students achieved similar scores in the pre- and post-tests (sometimes the score is even worse in the post-test, as in the case of the student 2 from the CG group and the student 7 from the EG group). Therefore, the overall performance of the students depended on the subtest. In Section 3.5, it was mentioned that for sub-tests 2 and 3, mental rotations of objects are required. This finding suggests to the authors that mental rotations should be more practised in the course if these sub-tests are intended for inclusion in the overall student evaluation (mental rotations were not directly studied in the current format of the course). Studying the individual student results in Fig. 14, it was observed that the EG’s performance in the post-tests of sub-tests 1 and 4 (Fig. 14 (a) and (d)) is clearly improved, since most of the points (which correspond to individual students) are above the diagonal line. In fact, only one and two students are on the line in the sub-tests 1 and 4, respectively (EG’s students are denoted as red circles). Note that in the sub-test 4 there are two students that have a score of 0% in the pre-test and have 66.7% and 16.7% in the post-test (see Table 3). In the CG, there are more students below the diagonal line. Figures 14 (b) and (c) show that the performance of EG and CG in the sub-tests 2 and 3 is similar in the pre- and post-tests. There are several points below the diagonal line, for both EG and CG groups. If the individual average scores are analysed in Fig. 15, it should be highlighted that most of the points corresponding to the EG results (denoted as circles) are located above the diagonal or on it, meaning that most of the students obtained a better score in the post-test than in the pre-test. In fact, only two students did not obtain a better score in the post-test (see students 6 and 9 in Table 3). On the other hand, in the case of the CG (denoted as crosses),


19

the performance of the students in the post-test is not always an improvement when compared with their respective performances in the pre-test. Fig. 16 shows that the CG obtains, in general, better average scores than the EG in the pre-test (dark boxes). This is evident in the following comparison, only the 35% of the students in the CG did not pass the pre-test, whereas 50% of the students in EG did not pass the pre-test. The minimum and maximum average scores in the CG’s case were 45.8 and 68.9, respectively (see students 7 and 5 in Table 3). In the EG’s case, the minimum and maximum scores were 39.0 and 69.7, respectively (see student 2 and 7 in Table 3). It is evident that the EG has a larger dispersion of the data than the CG, as can be seen in Fig. 16. Focussing on the post-test results (white boxes), the overall performance of the EG is, in general, better than the performance of the CG. Furthermore, the gain in the performance on the post-test is more remarkable in the EG. For instance, the whole EG passed the post-test, with a minimum score of 50.1% (student 6 in Table 3). Hence, there was a clear improvement in the spatial abilities of the students in EG who achieved relatively low scores in the pretest, compared to the CG. The maximum obtained score is 75.7% (student 7, EG), which is higher than the maximum score achieved by any student in the pre-test. In the CG, on the contrary, there were 2 students who did not pass the post-test: namely, student 2 (39.5%) and student 12 (38.0%). These students passed the pre-test with a score of 57.7% and 54.7%, respectively. Finally, from the results of the t-test summarised in Table 4, it can be concluded that EG’s difference of the means obtained in the pre- and post-tests is statistically significant in sub-tests 1 and 4, in which cases the computed p-values are 0.03 and 0.02, respectively. The CG students obtained similar means in these pre- and post-tests. The t-test gives p = 0.4 for the sub-test 1 and p = 0.6 for the sub-test 4, which means that the improvement in the performance of both post-tests is not statistically significant in the CG’s case. As can be seen in Table 4, the mean obtained in the sub-tests 2 and 3 are similar in the pre- and post-tests, for both the EG and the CG. These results were expected, since the scores obtained by EG and CG students in the subtests 2 and 3 were similar in the pre- and post-tests (as mentioned in the results presented above). The calculated p-values are > 0.05 in both the EG and CG cases. The last row in Table 4 shows the means and standard deviations of the average scores obtained by EG and CG students (last two columns in Table 3). Note that the average score is obtained by computing the mean of the scores obtained by each student in the 4 sub-tests. It can be seen that the students who participated in the robotics course (EG) showed a greater increase in their post-test mean scores compared to the increase shown by the students who did not join the robotics course (CG). In particular, the t-test gives a p-value of 0.07 in the EG’s case, while the calculated p-value is 0.61 in the CG’s case. Therefore, the difference between pre- and post-test means are not statistically significant for either of the studied groups, setting α = 0.05 as a significance level (5%).

20


It should be highlighted that the sample size (denoted from now on as N ) is very small in the current analysis (N = 9 in the EG’s case; N = 12 in the CG’s case). The calculated p-value would be smaller with a larger sample size and, consequently, the mean difference could be more statistically significant.

5 Conclusions The key objective of the paper was to analyse the use of an educational robotics course to develop spatial abilities in 12 year old students. In order to introduce robotics to the students, a course of robotics was prepared. The main point was to design and implement practical sessions, in which the students would engage in hands-on learning. As part of the research design, the students were randomly divided into an experimental group (EG), which participated in the robotics course, and a control group (CG), which did not participate in the robotics course. The main goal of the current paper was to evaluate if the students who joined the robotics course (EG) acquired a deeper understanding of relevant spatial skills compared to students who did not undertake the robotics course (CG). In order to evaluate the spatial abilities of both groups, a pretest and a post-test were prepared. The development of the pre- and post-tests took into account the most suitable and effective tests existing in the extensive research literature. The spatial skill tests were composed of 4 sub-tests. An extensive analysis of the scores obtained by the EG and the CG groups in the pre- and post-tests was carried out. The results show that the students who joined the robotics course (EG) demonstrated a greater increase in their spatial abilities compared to the increase demonstrated by students who did not participate in the robotics course (CG). The results also show that, in a fine-grained analysis of the overall performance of the students, there was a dependence on the specific nature of each sub-test. The students who participated in the robotics course (EG) definitely showed a positive mean gain score in the post-test of sub-tests 1 and 4, while they did not improve their overall performance in the post-test of sub-tests 2 and 3. In particular, the improvement in the performance on post-tests corresponding to sub-tests 1 and 4 was statistically significant in the case of the EG group (Table 4). Therefore, this study demonstrates clearly the importance of informed and well thought out selection of the instruments that can be used to evaluate the spatial abilities of the students. Future lines of research include working with a larger sample size. The results may be more statistically significant. Furthermore, it would be desirable to incorporate the robotics course in an annual school program in order to have time for more sessions. Finally, taking into account the multidisciplinary nature of educational robotics, the potential value of specific courses in robotics for learning other concepts or skills in the area of science, mathematics, engineering or technology should be studied and evaluated. It may be useful for students to learn such concepts in a practical and motivating way.


21

References (2000) Principles and Standards for School Mathematics. National Council of Teachers of Mathematics (NCTM) (2006) Massachusetts Science and Technology/Engineering Curriculum Framework. Resource document. http://www.doe.mass.edu/frameworks/scitech/1006.pdf, Accessed: January 2015 Bakker M (2008) Spatial Ability in Primary School: Effects of the Tridio Learning Material, Master Thesis of Psychology. University of Twente, Enschede Barak M, Zadok Y (2009) Robotics projects and learning concepts in science, technology and problem solving. International Journal of Technology and Design Education 19(3):289– 307 Barker B, Ansorge J (2007) Robotics as means to increase achievement scores in an informal learning environment. Journal of Research on Technology in Education 39(3):229–243 Benitti F (2012) Exploring the educational potential of robotics in schools: A systematic review. Computers and Education 58(3):978–988 Bers M, Portsmore M (2005) Teaching partnerships: Early childhood and engineering, students teaching math and science through robotics. Journal of Science Education and Technology 14(1):59–73 Contero M, Naya F, Company P, Saorn JL (2006) Learning support tools for developing spatial abilities in engineering design. International Journal of Engineering Education 22(3):470–477 Coxon S (2012) The malleability of spatial ability under treatment of a first lego league-based robotics simulation. Journal for the Education of the Gifted 35(3):291–316 Datteri E, Zecca L, Laudisa F, Castiglioni M (2013) Learning to explain: the role of educational robots in science education. Themes in Science and Technology Education 6(1):29–38 Ekstrom RB, French JW, Harman HH, Dermen D (1976) Kit of factor referenced cognitive tests. Educational Testing Service, Princeton, New Jersey Hegarty M, Waller D (2004) A dissociation between mental rotation and perspective-taking spatial abilities. Intelligence 32:175–191 Highfield K (2010) Robotic toys as a catalyst for mathematical problem solving. Australian Primary Mathematics Classroom 15(2):22–27 Humphreys L, Lubinski D, Yao G (1993) Utility of predicting group membership and the role of spatial visualization in becoming an engineer, physical scientist, or artist. Journal of Applied Psychology 78(2):250–261 Metz S, Donohue S, Moore C (2012) Spatial skills: a focus on gender and engineering. Resource document. http://www.engageengineering.org/associations/11559/files/ ARP_SpatialSkills.pdf, Accessed: June 2014 Papert S (1980) Mindstorms: Children, computers, and powerful ideas. Basic Books, New York Rocard M, Csermely P, Jorde D, Lenzen D, Walwerg-Henriksson H, Hemmo V (2007) Science Education Now: A Renewed Pedagogy for the Future of Europe. Publications Office, Luxembourg Rockland R, Bloom D, Carpinelli J, Burr-Alexander L, Hirsch L, Kimmel H (2010) Advancing the Ë¨ in K-12 STEM education. The Journal of Technology Studies 36(1):53–64 Rogers C, Portsmore M (2004) Bringing engineering to elementary school. Journal of STEM Education 5((3, 4)):17–28 Sorby SA (2009) Educational research in developing 3-d spatial skills for engineering students. International Journal of Science Education 31(3):459480 Sutton K, Williams A (2007) Spatial cognition and its implications for design. International Association of Societies of Design Research, Hong Kong, China Sutton K, Heathcote A, Bore M (2005) Implementing a web-based measurement of 3d understanding. In: Computer-Human Interaction Special Interest Group Conference (CHISIG), pp 23–25 Verner I (2004) Robot manipulations: a synergy of visualization, computation and action for spatial instruction. International Journal of Computers for Mathematical Learning


22 9:213–234