Analysis of the added value for the quantitative reasoning competency at the Luis Amigó Catholic University in 2021

This work uses statistical techniques to determine the value added to the Quantitative Reasoning competence of the undergraduate students of the Universidad Católica Luis Amigó, Medellín, Colombia, in 2021. The statistical comparison is made graphically on a Cartesian plane by quintiles using the standard deviations of the Saber11 test and the institutional test scores taken in the fifth semester of undergraduate students. The value added is observed in the first two quintiles only, where 40 % of the lowest qualifications accumulate; however, in this study, it increases by 4 % in the top 20% of scores. Also, a predictive model for this competence is developed, and linearity, normality, independence of residuals, collinearity, and homoscedasticity are verified. Finally, this model shows acceptable results in the quantitative reasoning competence of the Universidad Católica Luis Amigó students.

Analysis of the added value for the quantitative reasoning competency at the Luis Amigó Catholic University in 2021

Introduction
The great challenges in current education include supporting students and providing tools to families to improve student learning.
The challenge of the Teaching Centers (schools and universities, among others) is to provide an inclusive and quality education (Moriña, 2017).
After the COVID-19 pandemic, the University is committed to articulating technology with pedagogical processes and improving each student's competencies when graduating from an academic program.Quantitative Reasoning is among the basic competencies of a professional (Upmeier zu Belzen et al., 2021).According to Budhathoki (2022), Quantitative Reasoning (QR) holds that: (…) QR is an individual's ability to understand and use numbers in context.QR includes the skill (a) to read, understand, interpret, and analyze numbers, quantities, and data presented in various forms; (b) to solve realworld problems using mathematical and statistical knowledge; and (c) to represent and model quantitative situations, to reason using quantitative evidence, to draw conclusions, and to communicate these conclusions.(p.16) Based on the above, Universities must enable improvement actions in this competence based on the analysis of indicators to have the evolution of each student during their process (Schmidt et al., 2023).However, measuring the value-added in undergraduate education can be a daunting challenge since wide variables are considered in the analyses (Lyons and Polychronopoulos, 2020).Also, in recent years, the learning outcomes achievement as a value-added of the education institutes has gained considerable importance in the experience at higher education.The student's educational experience, as well as the knowledge related to quantitative reasoning competence, can be measured through a wide range of tests such as Arithmetic Reasoning (AR) or Mathematical Knowledge (MK) (Tirre and Peña, 1993).Throughout the literature, there is an ongoing discussion of how to approach quantitative reasoning in the different areas of knowledge.
The analysis has focused on generating playful learning spaces that make it possible to share the improvement of this competence in students (Ozaltun-Celik, 2021;Lewkowycz et al., 2022).
There are qualitative and quantitative studies as well as assessments of Quantitative Reasoning (QR) done in the literature; however, there is a lack of research to provide independent variables related to QR and the possibility of extending their analysis to other years, which makes not possible to give an overall picture of this competence after and in the middle of the bachelor's degree education.The above allows to report and explain an institution's value.Different types of studies have been made in the literature to evaluate quantitative reasoning through precalculus (Bowers et al., 2017), calculus (Ozaltun-Celik, 2021), environmental science (Mayes et al., 2014), and language (Lewkowycz et al., 2022).
Nowadays, quantitative reasoning is a topic in constant research to improve basic skills and competencies in mathematics.Gallardo-Pérez et al. (2021), presented an added-value study on basic mathematics and physics science in engineering careers.Schmidt et al., 2023), the Teaching Centers (TCs) present indicators in their reports of the value-added of different competencies such as Quantitative Reasoning (QR).Also, these institutions are interested in understanding if a variable presents or not a significant dependence on QR competence.For these reasons, the aim of this work is to provide a statistical application to compute the value added for QR competence, which other TCs might use in their reports.
The Universidad Católica Luis Amigó carries out a process of value-added analysis, measured from the QR competence from 2018 to the present.This work performed a statistical analysis for students in the fifth level of the Medellín campus.The study also used the results from Saber11 tests applied by ICFES (Instituto Colombiano para la Evaluación de la Educación), in 2021.The analysis allowed us to identify the value added by the institution for a specific competency.
The latter made it possible to provide further improvement activities.

Methodology
The approach used in this work is a quantitative investigation, also known as Empirical-Analytical Research, due to the fact that Value-Added at the University requires indices, percentage distributions, and quantitative indicators that allow measurement (Montoya & Cogollo, 2018).
Likewise, the design is non-experimental, longitudinal, and correlational because variables are not controlled.This study took the variables in the natural context, and the data will be collected for different periods of the presentation of Saber11 tests (Oschwald et al., 2020;Hu et al., 2020).Figure 1 shows the four steps used in the methodological process to calculate the valueadded for Quantitative Reasoning (QR).The four steps described in Figure 1 are explained as follows.
Step 3: Statistical analysis to quantify the value added by the institution.The statistical analysis used quintiles (Bonal & González, 2020;Montgomery, 2020), as the statistician to quantify the value added of QR.Quintiles (Q) distribute the score in 5 equal portions: Q1, which groups the 20% of the lowest results, and Q5, which groups the 20% highest.The quintiles were estimated by following the proportion of the scale in each of the tests.The comparison of the results of the Saber11 tests (Mathematics) with the institutional QR tests is carried out by typing and standardizing the results.
The Quantitative Reasoning (QR) value added by the Universidad Católica Luis Amigó in its students is estimated using the difference between the quintiles of the Institutional test and the Saber11 test.The crossing of the results of these two tests gives origin to a relationship in the Cartesian plane, explained by four quadrants.The QR test will be analyzed as a dependent variable, and independent variables, or covariates, may intervene in the test provided at the University.The students' sociodemographic characteristics were taken as covariates, establishing the link with the institutional results.Table 1 shows each variable and covariates, their description, and the range of values it can take.Step 4: Statistical Modeling of Quantitative Reasoning.This step aims to develop a model to predict QR in years after 2022 and also to know whose variables are of greater significance in QR modeling.Firstly, in the development of the model, those data where students "clicked through" all the test questions in less than 2 minutes and where the students did mark no questions were not considered.Secondly, the model's parameters were fitted using only the data for the year 2018.The estimation of the score of the Quantitative Reasoning (QR) test was carried out using the multiple linear regression model described in Equation 1.The independent variables (X1, X2, …, Xn) were defined as those that presented an association in the correlation matrix; however, other variables that did not show associativity with the QR score could also be incorporated when running the model. (1) In Equation 1, Y corresponds to the Dependent Variable related to the score in the QR test performed at the university, β0 corresponds to the value of the score in the absence of independent variables, βi indicates how much the score changes due to the independent variable, and Xi corresponds to the independent variables.The proposed linear combination equation in terms of independent institutional characteristics is already shown in the literature (Rodgers, 2007;Fernandes et al., 2020).Thirdly, the Williams plot is constructed to identify possible outliers' points for the model and the prediction for other years (2018 to 2022).The leverages are developed using the diagonal elements of the Hat Matrix.This matrix is constructed in terms of the data matrix Χ and the Jacobian matrix of the four adjustable parameters of the linear model (βi).The matrix Χ is two-dimensional, depending on the number of data points and input parameter variables.Also, the ± 3 values of the standardized residuals are provided as recommended values reported in the literature (covering 99% of scattered data) (Atashrouz et al., 2017;Kar et al., 2018).

Sample selection and characterization results
Table 2 presents the total number of students who participated in the institutional test and who will be considered for this study.It should be clarified that for this work, the students approved to be part of the study and answered the questions freely from the Virtual Campus (Campus virtual, 2023).Despite the response obtained by students in 2021, participation has decreased compared to 2020.This Table shows that the participation percentage is low and below 47%.Therefore, the year 2020 is the one with a higher level of participation (47 %), and 2019 is the year that presents the lowest participation percentage (23%).Figure 2 illustrates the sociodemographic and economic characteristics of the students who took the institutional test.Around 70 % of the students presented the Saber11 test between 5 and 9 years ago, while the rest took it more than 9 years ago.In Medellín, students in the 5° level of undergraduate programs belong to stratum 3 (45%), followed by stratum 2 (34%).Most of them Analysis of the added value for the quantitative reasoning competency at the Luis Amigó Catholic University in 2021 live with their parents (father and mother), and as a second option, they do so with relatives other than their parents.When analyzing the income received by the family group, it is observed that most families do not exceed USD 400,90.
The work activity of the students reflects economic dependence, and the parents' education level focuses on primary and secondary education.54% or more of the parents reach an education level in primary and high school, while the professional level ranges between 9% and 34%.The mother's education level is similar to that observed in the father.When analyzing the parents' marital status, a slight decrease in the percentage of separated parents is evident concerning couples with stable relationships.
The elapsed time of presentation of the Saber11 test concerning the moment of presenting the institutional test by the students is considered distant in all the cases.The time difference is between 4 and 9 years.The results of the QR test show low levels in the skills associated with analysis and logical reasoning in the institution's students.The arithmetic average of the quantitative reasoning test is 2,3, while in the mathematics competition in the Saber11 test, it is 52 points.The average measured on a scale from 1 to 5 does not present approval levels (assuming a rating of 3,0).The behavior in the mathematics test presented in Saber11 is similar.The results are low on a scale of 1 to 100 (assuming 60 is an acceptable score).The value added for the QR was calculated from the difference between the scores obtained in the Institutional test and the Saber11 test.It is important to note that the Saber11 test's scale varies from 0,0 to 100 (ICFES, s.f.) while the QR institutional test varies from 0 to 5. The analysis was carried out using quintiles.Figure 5 compares the percentages of students in the quadrants from 2018 to 2021.A slight increase in Quadrant I is shown, however, it makes a small contribution to value added.In Quadrant II a stable behavior is shown, and it becomes representative as it reflects the value added by the students.In the case of Quadrant III, the decrease in percentage is beneficial since the number of students with low scores in the Saber11 Mathematics test and the QR institutional test is reduced.In contrast to quadrant IV, where the percentage decrease in the last year reflects a reduction in students entering the institution with Saber11 Mathematics scores above the average, and in the institutional QR test, the score is below average, which is beneficial for the institution.Rodríguez-Revilla and Vallejo-Molina (2022), determined the value added of QR of four Higher Education Institution (HEI) groups related to (1) accredited and official, (2) accredited and private, (3) non-accredited and official, (4) unaccredited and private.These authors employed standardized mean indicators from 2012 to 2016.When the authors compared all the analyzed groups, they found that the private HEIs provided a value-added greater than the public HEIs in QR, regardless of whether they were accredited.These authors concluded that applying other statistical tests (different from the arithmetic mean) must be necessary to draw better conclusions above the value added.In comparison, our work shows that QR in the students increased only 4 % in the top 20% of scores using a quintile standardized analysis applied in the middle of their Curriculum.

Proposed model for Quantitative Reasoning
Table 4 shows the correlation matrix to identify the relationship between the student's QR com-petence with sociodemographic and academic variables.Also, the Critical Reading (CR) test and QR were included in the analysis (Awofala and Lawal, 2022).In this table, the symbol ** indicates that the correlation is significant at the 0,01 level (bilateral), and the symbol * indicates that the correlation is significant at the 0,05 level (bilateral).The knowledge of this relationship will make it possible to propose a statistical model for prediction processes.Two models can be developed using the multiple linear regression method.The model incorporated the independent variables that presented an association with the QR test.In conclusion, the model to estimate the QR score at the institutional level is represented by Equation 2: (2) In Equation 2 The normality test was performed using the P-P graph.This graph observes the adjustment of the residuals with a linear pattern.Then, employing the Kolmogorov-Smirnov test, whose P-value or significance is greater than 0,05, the normality of the residuals is guaranteed, as seen in Figure 7 and Table 7.In this table, the superscripts a and * indicate the Lilliefors significance correction and the lower bound of the true significance.The assumption of independence is because the residuals or errors are independent between them.According to the Durbin-Watson statistic, the residuals generated for the QR test are independent (it takes a value of 1,847, which is close to 2).Then, the collinearity analysis uses the Variance Inflation Factor (FIV) factor, which must be close to 1 to guarantee non-multicollinearity.The variables that explain the model do not represent linear dependence between them, i.e., there is no multicollinearity because the Variance Inflation Factor (FIV) is close to 1. Finally, the assumption of homoscedasticity is analyzed in the scattergram of typified forecasts and standardized residuals.Figure 8 shows the standardized residual of QR results.As seen in this figure, uniform variation is observed and does not present a defined pattern in its distribution.Therefore, it achieves the assumption of homoscedasticity.Another correlation analysis is done to evaluate if the model constructed in 2018 can be able to predict other years and used for comparison purposes.Figure 9 shows the Williams diagram.This plot is done with the model provided in 2018 and is extended to predict other year behavior.This diagram provides an overall picture of possible outliers of the model when it is applied to describe other conditions different from the fitting procedure.The 0,70 % of the data present ± 3 above standardized residuals while the 0,75 % presents data above the critical leverage (h* = 0,009 where five constant parameters and 2012 data after data cleaning), demonstrating low outliers' values.The above shows that the level of students in the QR is currently maintained.With the final model, the authors estimate the average relative deviation for 2018, 2019, 2020, and 2021 with values of -0,3 %, 10.96 %, -2.50 %, and -3.95 %.Low values trend to zero, demonstrating satisfactory distribution between positive and negative errors.In conclusion, the QR results have remained constant over the years, a topic that must be improved in the University to increase QR scores.

Conclusions
The value-added analysis of the fifth-level students of Universidad Católica Luis Amigó can highlight some aspects as a starting point for academic planning and improvement plans.At the national level, more than 56 % of students finished high school in public education institutions.Most belong to socioeconomic stratum 2 or 3, and some work.Also, students live with their parents; few do with family or friends.Regarding the father and mother, the education level is between the primary and high school levels.The income of the family group does not exceed two million Colombian pesos per month.
Concerning the results obtained in the Saber 11 tests in Mathematics and Language, they were within the Acceptable level of 60 points on the 100 scale (46 for Mathematics and 52 for Language).The same situation occurred in the institutional test, where the acceptable level was 3,0 on the 5,0 scale, and the results were 1,7 for QR.The value added for the institution took relevance in 2021, although the reflected contributions are in the first quintiles (Q1 and Q2), where the lowest scores accumulate.In the same year (2021), the negative values 2018 in the top quintile (Q5) in QR and Critical Reading skills were also overcome.
The real QR value added is shown in Q5, where the highest score set can be found (above 80 %), and it only increased by 4 %.Finally, a linear Gabriel Jaime Posada-Hernández Mauricio López-Bonilla Diego Alejandro Uribe-Suarez Luis Fernando Cardona-Palacio model was developed to determine the QR relations.The model was also extended to predict other years.The model results showed low outliers percentage data of the QR sample.Also, Critical Reading and Language significantly contributed to the QR competence.Other institutions can employ the statistical analysis applied in this work in order to provide QR value-added and subsequently compare it to the results of this work.Further work is needed to report the added value other relevant competencies, such as reflective reading and learning outcomes.

Figure 1 .
Figure 1.Four steps used to calculate the value-added for Quantitative Reasoning (QR).

Figure 2 .
Figure 2. Socioeconomic characterization of the students who took the institutional test in 2021: a) Socioeconomic stratum, b) student coexistence, c) family income, d) education level.

Figure 3
Figure 3 shows the distribution of quintiles for the years 2018 to 2021.In this figure, the Q shows similar value-added behavior over the years.Q1, Q2, and Q5 exhibit slight positive increases; meanwhile, Q3 and Q4 exhibit slightly negative

Figure 3 .
Figure 3. Value added by Quintile statistical analysis of the QR from 2018 to 2021.

Figure 4 .
Figure 4. Analyses of quadrants in the value-added % QR in the year 2021.

Figure 5 .
Figure 5.Comparison of the value added by quadrants (I, II, III, IV) from 2018 to 2021.

Figure 6 .
Figure 6.Linearity of the QR with the independent variables.

Figure 7 .
Figure 7. Normality for the residuals of the QR.

Figure 8 .
Figure 8. Scatterplot for the residuals and forecasts for the QR variable.
Figure 9. William's diagram of the QR model.

Gabriel Jaime Posada-Hernández Mauricio López-Bonilla Diego Alejandro Uribe-Suarez Luis Fernando Cardona-Palacio 2021;
These authors employedPérez et al.,d pre-test analysis to correlate the value added.These correlations were in the first four semesters of the elemental training cycle.The latter included, among others, subjects such as differential calculus, integral calculus, vector calculus, and differential equations.The results showed that courses in differential equations, linear algebra, and classical mechanics contribute more to other variables.betweenadvanced courses and no prior education in business and economics.According to the literature(Franco-Gallego, 2019;Pérez et al.,

Analysis of the added value for the quantitative reasoning competency at the Luis Amigó Catholic University in 2021 Step 2: Data
ates students pursuing the 5th semester of a bachelor of 10 semesters in terms of QR.The institution designed the test considering the competencies established in the SaberPro test managed by the ICFES (2023).The Basic Science Department at Universidad Católica Luis Amigó was in charge of performing the test.This test evaluated quantitative reasoning skills concerning numbers, arithmetic operations, and graph analyses.It is also worth mentioning that the designed test examined statistics, geometry, algebra, and calculus (ICFES, 2023).The test performed at the institution characterized the socioeconomic conditions of fifth-level students from the Medellín, Colombia campus using the variables described in step 1 (Universidad

Table 1 .
Variables and covariates for the QR test.

Table 2 .
Student participation in the institutional QR test.

Gabriel Jaime Posada-Hernández Mauricio López-Bonilla Diego Alejandro Uribe-Suarez Luis Fernando Cardona-Palacio the
QR Institution test.This group represents 20,2% of the population at the university.The idea is to keep this group with minimum values.
This group includes 32,9% of the institution's students.It would be expected that this percentage is high in order to maintain a good level of generic QR competence in students.corresponding to 19%.As the percentage in this group increases, the impact of the value added becomes more visible to the institution.In conclusion, the real value added is shown in the second quadrant (II) since the student's institutional test score is above the Saber11 test competencies (the x-axis is located at the left of the origin).

Table 4 .
Correlations matrix regarding the CR and QR skills.

Table 5
Gabriel Jaime

Table 6 .
Constants, unstandardized coefficients, and typified coefficients of the evaluated linear regression models.

Table 7 .
Normality for the residuals of the QR.