The workshop on CALT Release 1.6 was held between 13-17 December 2021. The objective was to discuss several issues that arose during the development of the previous CALT Release – 1.5 and to present the latest improvements concerning the software, while considering the essence of the BILC working group report, on (1) test items, (2) instructions, (3) time, (4) interface, (5) algorithm, (6) teat security, (7) importance of the project. The received feedback from the participants showed a positive attitude towards a new mode of testing thanks to an innovative software design and a new approach to a testing procedure The improvements of the software design on hiding the fifth option “I don’t know the right answer” increased the effectiveness of the testing results because the test takers were not confused. This decision also influenced positively the statistical analysis which is provided by the software product. The necessity to increase the security level of the software product was stated.
It was proposed to introduce additional security codes for access to the program as well as to start the testing session to ensure its protection of the personal data of test-takers. It was also highlighted the necessity to improve the algorithm and method of testing to provide more adaptive testing sessions. To fulfill such a task, it is necessary to make the testing sessions shorter in time. Such a decision leads to the reconsideration of the sequence of the items in the algorithm. Considering that the sufficient level of the software product was reached, it had been recommended to share the CALT solution within NATO DEEP partner institutions. It was suggested to organize three-day virtual meetings with representatives of the NATO DEEP entities to familiarize with the theoretical foundations and practical implementation/ utility of the CALT. Day 1 – the theoretical part of the CALT solutions. Day 2 – a practical demonstration of the CALT software for further utility. Day 3 – Q&A and wrap up.