The aim of this study was to validate the Team-Based Learning Student Assessment Instrument in students of a Dental School of a Brazilian University. The validation process was performed progressively in the steps: translation and back-translation; consensus version and cross-cultural adaptation; face validation; teste version; test-retest. The reliability assessment was performed by calculating the internal consistency by obtaining the Cronbach Alpha coefficient. The statistical analysis of the test-retest was the calculation of the intraclass correlation coefficient. The margin of error used in the tests was 5.0%. After application of TBL-SAI to a population of 80 students, the Cronbach's Alpha coefficient was 0.670. After removing questions 12 and 19, the result would change to 0.702 and 0.701, respectively. The validation process was carried out in a satisfactory way. It was verified the need to reduce the quantity of items, in order to make the evaluation of perception in relation to the method of teaching more reliable.

Keywords: team-based learning, active methodologies, dentistry, validation

Education of the current generations (millenials) poses a challenge for teaching. These students’ value communication and teamwork.¹ Thus, traditional education, in which the teacher makes the deposit of contents, while the student must memorize them² does not fit the parameters necessary for an effective contemporary teaching. In this sense, active learning methodologies appear as a strong allied to meaningful teaching. They are educational methods capable of engaging students in the learning process, putting them in control of their own path of knowledge development.³ Team-based learning (TBL) represents one of these.⁴ This instructional strategy has been shown to be viable and effective in the professions of health and medical education.⁵⁻⁸ However, these investigations were applied in North American and Asian students, and the application of the perception assessment method in Brazilian students is non-existent. One of the instruments capable of analyzing students' perceptions about TBL is the Team-Based Learning Student Assessment Instrument (TBL-SAI), developed by Mennenga, in 2012.⁸ However, to apply it in a Brazilian population, whose primary language is Portuguese, a validation study is necessary. Validating an instrument means obtaining a new version similar to the original, making cultural adaptations, as well as confirming properties that show the ability to measure it. This study had the objective of validating the TBL-SAI in students of a Dental School of a Brazilian University.

Study design and population

This was a validation study of the TBL-SAI questionnaire,⁹ originally written in the English language. The population was composed of students over 18 years of age, of both sexes, who attended the Dental School of Federal University of Pernambuco (Brazil) in 2018, which used TBL as teaching methodology. This research was approved by the Institutional Research Ethics Committee, under the number of process: 81177017.5.0000.5208. Subjects who agreed to participate were included, by reading and signing the Free and Informed Consent Form.

TBL-SAI questionnaire

The questionnaire presents 33 short and direct multiple-choice questions whose answers are on a Likert scale, in which all questions should be answered based on agreement with the sentences that are stated in each question. For each sentence (question) a number is assigned according to the degree of agreement:

1. Strongly Disagree.
2. Disagree.
3. Neither disagrees nor agrees.
4. I agree.
5. Strongly Agree.

The TBL-SAI consists of three subscales:

1. Responsibility, consisting of eight items.
2. Preference for expositive/traditional class or ABE, containing 16 items.
3. Student satisfaction, comprising nine items. In this study, a printed version was used to validate the questionnaire.

Validation

Translation and back-translation

The TBL-SAI was translated into Portuguese by a bilingual professional native to the Portuguese language fluent in English, who knew the subject and questionnaire. The back-translation, whose purpose is to check whether the translated version reflects the original, was performed by another bilingual native English-speaking professional with no knowledge of the original instrument. It aids in the identification of possible inconsistencies.

Consensus version and cross-cultural adaptation

A multidisciplinary team consisting of two professionals, one graduate in dentistry and the other a graduate in business administration compared the two versions (translated and back-translated) with the original to obtain the version used in face validation. In this process, there was a need to adapt the term "lecture" whose consensus resulted in the translation: "traditional class" or expositive lecture”. It is common to change, create or delete terms considered inappropriate or ambiguous, as long as the concept is similar to the original.

Face validation

The face validation of TBL-SAI was performed with a group of 9 individuals. At this stage, the students answered the questionnaire individually and commented on the issues of the applied version, pointing out difficulties and suggesting more understandable terms. Respondents were asked to verbally paraphrase each item and the evaluators wrote down, to verify whether or not the item was understood. Subsequently, modifications were made necessary for a better understanding and adaptation of the questionnaire to the target audience. Finally, it was also recorded the average time each participant took to answer it in full, besides registering the acceptability of the TBL-SAI format.

Test version

Based on the performance of the questionnaire in face validation, in relation to the understanding of the questions and also in the suggestions of the respondents, the examiners met to make the necessary changes, which led to the writing of the test version of the instrument (Table 1). The elaborated version was later tested for its psychometric properties and forwarded to the authors of the original questionnaire for the conference.

Test-retest

At this stage the questionnaire was applied to a sample composed of 30 individuals (test) at the dental school of UFPE. After 7 days, the questionnaire was re-applied to the same individuals to verify agreement, aiming to verify the reliability of the questionnaire and its consistency.

Statistical planning

The data were compiled in Excel and the statistical analysis was performed using MEDCALC software in version 14.8.1. The margin of error used in the tests was 5.0%. The reliability assessment was performed by calculating the internal consistency by obtaining the Cronbach Alpha coefficient, which is one of the most used psychometric indicators to verify reliability or internal consistency of an instrument. The statistical analysis of the test-retest was the calculation of the intra class correlation coefficient (ICC), adequate parameter to define the stability of the instrument in question. The ICC can range from 0 to 1 and is considered a reflection of the agreement or correlation between the scores obtained in the test and the retest by a group of evaluated individuals. The reference values are: ICC less than or equal to 0.40=weak correlation; ICC between 0.41 and 0.60=moderate correlation; ICC between 0.61 and 0.80=good correlation; KIC greater than or equal to 0.8=excellent correlation.¹⁰

The validation and adaptation process for the Portuguese language was linear. Only one modification was made in order to promote understanding regarding the questions of the form. This was done by replacing the term "lecture" with "traditional class" or “expositive lecture”. In the face validation, all participants reported to fully understand the questionnaire. There was no report on the need for modifications of the evaluation items. The Cronbach's Alpha coefficient was applied after the application of the Test to a population of 80 students. The result for the total of questions was 0.670. After removing questions 12 and 19, the result would change to 0.702 and 0.701, respectively. This would make the result acceptable. The questionnaire was applied again with 30 students (retest). The degree of agreement between the two evaluations was obtained from the ICC and respective confidence interval between the two evaluations for each of the statements or item is expressed in Table 2.

Dental education increasingly requires the use of active methodologies suitable for the training of future professionals. In this sense, the ABE method has a much greater potential to prepare students for the professional challenges of the 21^st century when compared to the traditional teaching model. In this traditional model, there is a unidirectional knowledge distribution of a teacher to the students.² However, to verify the appropriateness of the method to its actual application, it is necessary to evaluate the perception regarding the use of the method, as proposed by Mennenga.⁹ This research was a validation of the TBL-SAI research instrument for a Brazilian population, whose results revealed a good degree of understanding by the respondents. Few adaptations were performed to the questionnaire for Portuguese-speaking students.

In this sense, an analysis was performed to ascertain the intensity of correlation between the items of the questionnaire-Cronbach's Alpha coefficient. This coefficient may increase after the deletion of an item from the measurement scale (questionnaire). If this occurs, it is assumed that this item is not highly correlated with the other items on the scale. Therefore, Cronbach's alpha determines whether the scale is actually reliable because it evaluates how each item reflects its reliability. Thus, the removal of items 12 and 19 provided this change in the application of TBL-SAI in this investigation. It is important to note that the alpha value is affected by the number of items that make up a scale. As the number of items increases, the variance, systematically placed in the numerator, is increased such that an overestimated value of the consistency of the scale is obtained.¹¹ Similarly, the value of Cronbach's alpha can be overestimated if the size of the sample is not taken into account: the larger the number of individuals who fill a scale, the greater the expected variance.¹² The ICC calculation revealed a weak correlation between test and retest for questions 7, 8, 11, 14, 17, 19, 20, 23 and 30. This result can be attributed to the fact that TBL-SAI is an extensive form of questions, which would make it difficult to obtain similar results after a period of time. Moreover, since the Likert scale has very close answers, this divergence could be expected to some extent. These data point to the need to adapt the questionnaire, with the possibility of reducing the number of questions, or even reformulation, in order to identify items that could be aggregated into unique questions. From this perspective, we suggest that a simplified TBL-SAI, with 27 questions, be applied to future students of Brazilian dental schools. In this sense, items 7, 8, 11, 12, 19 and 30 would be withdrawn. In addition, future work is needed in order to establish this proposal of evaluation of students' perceptions about TBL, which has shown to be stimulating for the students present a more active position in the search for knowledge, obtaining a greater involvement with the studied subject.^6,13,14

The validation process of the Team-Based Learning Student Assessment Instrument for a group of Brazilian students of a dental school was carried out in a satisfactory way. It was verified the need to reduce the quantity of items, in order to make the evaluation of perception in relation to the method of teaching more reliable.

None.

The authors declare that there are no conflicts of interest.

1. Boysen PG, Daste L, Northern T. Multigenerational challenges and the future of graduate medical education. Ochsner J. 2016;16(1):101−107.
2. Mitre SM. Metodologias ativas de ensino-aprendizagem na formação profissional em saúde: debates atuais. Ciência & saúde coletiva. 2008;13(2):2133−2144.
3. Prince M. Does active learning work? A review of the research. JEE. 2004;93(3):223−231.
4. Bollela VR. Aprendizagem baseada em equipes: da teoria à prática. Medicina. 2014;47(3):293−300.
5. Abdelkhalek N. Using team-based learning to prepare medical students for future problem-based learning. Med Teach. 2010;32(2):123−129.
6. Livingston B, Lundy M, Harrington S. Physical therapy students’ perceptions of team-based learning in gross anatomy using the Team-Based Learning Student Assessment Instrument. J Educ Eval Health Prof. 2014;11:1.
7. Takeuchi H. Effects of team-based learning on fixed prosthodontic education in a Japanese School of Dentistry. J Dent Educ. 2015;79(4):417−423.
8. Addo-Atuah J. Performance and Perceptions of Pharmacy Students using Team-based Learning (TBL) within a Global Health Course. Innovations in Pharmacy. 2011;2(2):1−15.
9. Mennenga HA. Development and Psychometric Testing of the Team-Based Learning Student Assessment Instrument. Nurse Educ. 2012;37(4):168−172.
10. Li L. Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy-based brain imaging. J Biomed Opt. 2015;20(5):50801.
11. Krus DJ, Helmstadter GC. The Problem of Negative Reliabilities. Educational and Psychological Measurement. 1993;53(3):643–650.
12. Bland J, Altman D. Statistics Notes: Cronbach’s Alpha. BMJ. 1997;314:572.
13. Haj-Ali R, Al Quran F. Team-based learning in a preclinical removable denture prosthesis module in a United Arab Emirates dental school. J Dent Educ. 2013;77(3):351−357.
14. Simonson SR. Making students do the thinking: team-based learning in a laboratory course. Adv physiol Educ. 2014;38(1):49−55.