Computational Thinking: A Proposed Formative Assessment Rubric for Physical Computing Courses

The term Computational Thinking (CT) is commonly acceptable as a 21st century skill in reading, writing, and arithmetic, causing many states to adopt new policies as far as the curriculum, didactic material, teaching and learning methods, as well as assessment methods. In this frame, we have developed a series of Arduino lessons for teachers’ training, including physics computing and computational thinking development. For the training material quality improvement, we have developed a set of assessment criteria (Rubric) that will be used during the teacher’s training course resulting in the formative assessment. The aim of this work is to analyze and justify the rubric’s format and priorities, as well as to inform the teachers’ trainers about the course assessing priorities.

Introduction

Although the term “Computational Thinking” has not yet reached a global consensus agreement, the majority of researchers and policymakers perceive it as a thought process that is involved in designing solutions that a computer and/or a human can execute. This conceptualization is based on the definition devised by Janette Wing [1]–[3]: “Computational thinking is the thought processes involved in formulating a problem and expressing its solution(s) in such a way that a computer—human or machine—can effectively carry out”. Wing also declared that CT is a fundamental skill like reading, writing, and arithmetic and must be taught to everyone and not only to those who plan a career in the CS or STEM field [1], [4]. Fundamental issues for STEM’s transdisciplinary approach in different knowledge objects include CT research [5]–[7]. Moreover, Computational Thinking concepts using the Computational Pedagogy Model in STEM, constitute a field research for education [7]–[9].

An updated definition of CT which involves Computing is given by the Digital Education Action Plan 2021–2027 [10]: Computational thinking, programming, and coding are often used interchangeably in education settings, but they are distinct activities. Programming refers to the activity of analyzing a problem, designing a solution, and implementing it. Coding means implementing solutions in a particular programming language. Computational thinking, shorthand for “thinking as a computer scientist,” refers to the ability to understand the underlying notions and mechanisms of digital technologies to formulate and solve problems. Generally, the most dominant definitions concerning CT core include concepts, such as abstraction, algorithmic thinking, automation, decomposition, and generalization [11].

The above skills are very critical for everyone in order to participate in the digital society and/or for their professional development [3]. The challenge of teaching and learning CT skills pushed many educational systems to integrate CT skills into curricula [12]. Many open hardware platforms (such as Arduino, microbit, and Raspberry pi) and education robotics constitute tools in Education 4.0 [13]–[17]. In Greece, similar initiatives exist [18]–[20]. Currently, a course is designed for teachers and trainees. The aim of the course is to improve skills in physical computing using the Arduino platform and Ardublock (open-source software) to solve problems in the real world. The lessons were designed to introduce: physical computing, the fundamentals of sensors and actuators, the algorithm definition, the logical connective, the if-then-else, and the while statements [21].

But, although there have been developed too many didactic materials, teachers suffer from a lack of CT assessment and/or evaluation material [22]. Thus, the present work represents a proposed assessment rubric that allows teachers as trainees to be assessed. The effectiveness and the usability of the proposed rubric will be examined during teachers’ training at Open Lab at the Regional Unit of Viotia under the programs of Annual Pedagogical Training Program (EPPAIK) in Levadia departments of the Higher School of Pedagogical and Technological Education (ASPETE).

Related Works

The majority of assessment tools and methods are based on the Rubric schema formulated by [23]. Also, the evaluated concepts, which are analyzed to criteria and scales, are based on Computing and aim to depict the development of CT concepts [24]. Selby’s work is limited to abstraction, decomposition, algorithmic thinking, evaluation, and generalization [25]. Another important work, where the CT assessment based on a four interconnected stages framework, presents the procedure of developing an assessment approach for a proposed lesson in the classroom [26]. This case study shows a clear way of assessing CT concepts through a lesson. An alternative approach to CT assessment was proposed by Brennan and Resnick who stated that CT is strongly connected with a set of attitudes and skills (practices) that involves the creation of computational artefacts, debugging, testing, collaboration, and creativity [27].

The Proposed Rubric

The present work proposed a holistic rubric focusing on the assessment of didactic material offering activities based on Computing and STEM. Even though the proposed rubric concepts are based on Selby’s work [24], it is actually an innovative rubric because the included criteria are concerned with domain-specific area CT definitions (such as the offered Arduino/STEM course, Table I). This area requires specific knowledge or skills in order to solve problems in the subject of Computer Science or Programming [28], [29]. Thus, it has followed a framework similar to that has been proposed by Tang [30], which separates CT definitions into computing concepts and competencies.

Table I. Sessions of Course for Physical Computing Courses Using the Arduino Platform for Formative Assessment

Session	Hours	Learning objective
Introduction to physical computing	1	The role of microprocessors in the physical world The analog-to-digital conversion.
Fundamental of common electronics components	2	Introduction to electronic components (LEDs, resistors, switches, and potentiometers) and basic electrical connections using wires, breadboards, connectors, and components.
Fundamentals of common sensors	2	Introduction to common sensor components (LDR, IR sensor, ultrasonic sensor, etc.) and basic electrical connections.
Fundamentals of common actuators	2	Introduction to common actuator components (servo, dc motor, etc.) and basic electrical connections.
Programming by solving a real problem	4	Traffic light and rail crossing using an automatic barrier: The aim of the project (like in the real world) is to safely pass a car over a rail crossing by using basic programming (loop, cases, and digital I/O) using electronic components, an ultrasonic sensor, a LED, and a servo. The artifact which the trainers develop is shown in Fig. 1.
Programming by solving a real problem	4	Climate control in a greenhouse: The aim of the project is to control the climate inside a greenhouse by using basic programming (loop, cases, and digital I/O) using electronic components such as LDR, temperature and humidity sensors, LEDs, and servos. The artifact which the trainers develop is shown in Fig. 2.

Fig. 1. Artifact for project traffic light and rail crossing using an automatic barrier.

Fig. 2. Artifact for project climate control in a greenhouse.

The term “Assessment” is used for the trainee’s judgment performance in relation to specific goals, and a formative direction requires (a) feedback and (b) an indication of how the work can be improved to reach the required standard [31]–[33]. Using the term “feedback” we adopted the definition of Ramaprasad [34], [35]. He describes feedback as the distance between the actual and the expected result, and this is subsequently used to alter the gap in some way. Generally, there are two types of rubrics: (a) the “analytic rubric” in which each criterion is evaluated separately, and (b) the “holistic rubric” in which all dimensions are assessed simultaneously to provide a single overall score [36]. The use of rubrics in the educational process offers many benefits to both trainees and trainers: (a) inform trainees of expectations, (b) provide feedback, (c) maintain consistent grading and fair assessment, and (d) enhance trainees learning and self-assessment [37]. In addition, rubrics provide trainers with mechanisms to (a) clarify teaching and learning goals, (b) analyze trainee’s scores with specific criteria and skills, (c) summarize trainee’s performance reliably, and (d) identify patterns of strengths.

The proposed rubric as shown in Table II has been structured in 4 levels so that it is possible to identify not only the existence or use of the concept but, mainly, to measure a qualitative value for its application. Moreover, the odd number of levels, for example, 3 or 5, usually prompts the evaluators to opt for the middle option instead of their neighbor, which usually depicts a wrong result. On the other side, the number of 5 levels would give unnecessary information to evaluators.

Table II. The Proposed Rubric

Criteria	Level-1	Level-2	Level-3	Level-4	CT concepts
Decomposes a problem into chunks of subproblems driving to a solution based on the subproblems’ solutions.	Ineffectively tries to notionally separate the initial problem.	Formulates a partly acceptable decomposed problem solution.	Formulates a series of individual subproblems which are part of the initial problem.	Designs solvable individual subproblems which results in the initial problem solution.	DE, AL, AB
Finds similarities/differences during a problem analysis (pattern recognition).	Cannot identify similarities and/or differences in a problem solution.	Identifies some of the similarities and/or differences in a problem solution.	Can transform the similarities to patterns.	Successfully applies the recognized patterns to the final problem solution.	GE, AL
Data, information analysis and evaluation	Cannot recognize the role of accurate data in a problem’s solution.	Partly effectively uses the notions of data and information.	Can collect, organize, store data, and evaluate the resulting information.	Evaluates the received feedback and improves the offered solution.	EV
Designs and creates artefacts	Inadequately designs artefacts.	Designs adequate artefacts but faces difficulties in the implementation phase.	Creates “weak” digital artefacts (lack of stability, trust worthless, not tested, etc.).	Creates solid digital artefacts, effective and well-designed.	AL, AB
Exploitation and usage of procedures	Has no clear image of a procedure usage.	Knows that a procedure can be used to hide the detail with subsolution.	Inadequately uses the procedures in a program solution (still needs improvement).	Perfectly designs, writes and debugs programs using procedures.	AL, DE, AB, GE
Propose solutions which are based on the solutions of smaller instances of the initial problem (Recursion).	Has no clear image of a recursion notion.	Cannot identify the recursion problem parts.	Ineffective uses the recursion (no results, high delays).	Perfectly uses the recursion.	AL, DE, AB, GE

The contribution of each criterion to the final assessment is equivalent. Each criterion has been opted in the scope of depicting at least one of the CT concepts. This is perceived as a more comprehensive way to holistically trainee’s assessment.

The abbreviations in Table II are: AB (Abstraction), DE (Decomposition), AL (Algorithmic Thinking), GE (Generalization), and EV (Evaluation). Moreover, is worth mentioning that each criterion includes/expresses more than one Computational Thinking concept. Fig. 3 quotes the proposed rubric as it is previewed in the Rubrics.io application which is used for the in-classroom teacher’s assessment needs. Also, each criterion is assigned a factor playing a more critical role in the production of the final mark assessment. For example, in our case, the first criterion has been assigned the factor 20% while the majority of the rest with 15%. The summation of all criteria should be 100%.

Fig. 3. The proposed rubric previewed in the rubrics.io application.

Fig. 4 depicts the rubric usage in practice as it is utilized by a trainer for a teacher’s formative assessment. For our research needs, we have created a demo teacher and we have assigned him/her two separate evaluations. The evaluations are presented in a visual way, thus making the results more perceivable and understandable by the evaluator (trainer).

Fig. 4. The proposed rubric in practice for a teacher’s formative assessment.

Future Work

The effectiveness and the usability of the proposed rubric will be evaluated by a sample of 90 students at OpenLab at the Regional Unit of Viotia under the programs of Annual Pedagogical Training Program (EPPAIK) in Levadia departments of the Higher School of Pedagogical and Technological Education (ASPETE). The results will be presented in a next paper.