Digitalised Language Proficiency Testing & Practice

Teaching and learning Arabic as a foreign language can be supported with the use of effective digital assessment tools. An Arabic extension to the C-Test[editor] has been added in order to facilitate the automatic generation of C-tests for assessing learners' Arabic while taking into account a range of language-specific features. This project comprises a Master thesis in Computational Linguistics in collaboration with LEAD, the Department of Oriental and Islamic Studies (Abteilung für Orient- und Islamwissenschaft) and the Center for Islamic Theology (ZITh) at the University of Tübingen as well as Humboldt University of Berlin and University of Würzburg. Additionally, an Arabic group has been formed for the development of C-tests which consists of academics from Tübingen and other universities, allowing for considering a variety of user-specific needs.

C-Test[editor]

The Arabic option of the C-Test[editor] includes a variety of language-specific features which address some linguistic challenges. The user is able to choose whether or not they would like to:

• include or exclude diacritics
• exclude the definite article (ال) from gapping (it is always connected to the word as a prefix, which may lead to invalid gapping as the article is sometimes equal in length to half of the length of the word to which they are connected, which results in incorrectly gapping the actual lexical item

Since Arabic is a highly inflectional language, a section is included in the C-Test[editor] which displays the stem of each gapped word.

Additional linguistic features are also included in the C-Test[editor], such as some linguistic complexity and readability metrics, as well as an overview of the number of verbs, adjectives, relative pronouns, etc. which can aid in measuring the difficulty of the text.

Language exercises

Work is currently underway to create exercises for second language learners of Arabic. These will include more general tasks such as fill-in-the-blanks, multiple choice, arranging words in the correct order, etc., as well as language-specific tasks such as filling in diacritics. The current correct/incorrect model of feedback will be expanded to include more elaborate feedback formats with hints to provide individualised assistance to learners.

Chinese is one of the most frequently learned languages in the world, so an effective Chinese proficiency testing tool such as the C-test is beneficial to both learners and teachers in helping identify learners' language proficiency levels. For learners, this means that they can self-assess their language proficiency and receive a recommendation for a course suitable to their individual proficiency level; for teachers this means the possibility to form homogeneous language groups. However, currently little research in this field has been done on non-European languages such as Mandarin Chinese. The C-Test[editor] has therefore been extended to also include Chinese in order to further research in this challenging area. A Chinese group has been formed for C-test development which consists of academics from the University of Tübingen and other universities. This group aims to meet the technical language-specific prerequisites for an automated placement test and discuss conceptual issues, including user-specific questions.

C-Test[editor]

The C-Test[editor] has been extended to include the automatic generation of C-tests in Chinese. The Chinese option provides users with language-specific settings, including:

• choosing whether or not to generate the test in simplified or traditional characters
• displaying testlets with or without Pinyin and optionally testing tone by number or mark
• selecting how many characters a gap should comprise

The gap unit in the editor is a single character rather than individual strokes. In other words, the gapped second half of a word consists of a complete character. The character-based deletion approach could help avoid testing only test takers’ literacy skills. Users are also provided with some linguistic complexity and readability statistics for each text, which are useful for the evaluation of the difficulty level of the text. Users can also see various statistics pertaining to the text, such as the number of lexical or grammatical items, repetitions, etc.

Additionally, the use of EBCL (European Benchmarks for the Chinese Language) vocabulary frequency list as a basic measurement of lexical difficulty is planned to be implemented in the future. By cross-comparing the words in the text with the EBCL list, users can easily see if their texts contain advanced lexical items that are beyond test takers’ learning scopes, and adjust the texts accordingly.

Language exercises

There are currently plans to develop language learning exercises with detailed and automated feedback for learners of Chinese who would like to keep track on their learning progress.

English has been implemented in the C-Test[editor] from the beginning, so most of the work lies in creating new features. C-tests for English have been tested for many years in both a paper format and electronically at the Foreign Language Center (FSZ) at the University of Tübingen, yielding a large collection of results for comparison. Currently, the C-Test[editor] offers a wide selection of text statistics for English, with multiple complexity and readability analysis features also currently in development.

French has been implemented in the C-Test[editor] from the beginning, so most of the work lies in creating new features. C-tests for French have been tested for many years in both a paper format and electronically at the Foreign Language Center (FSZ) at the University of Tübingen, yielding a large collection of results for comparison. Currently, the C-Test[editor] offers a wide selection of text statistics for French, such as an overview of the number of nouns, verbs, repeated gaps, etc. in the text.

C-tests in French have also been implemented as tests for general language proficiency assessment at the Department of Romance Languages at the University of Tübingen via the Ilias platform. An automated analysis on the results of the placement tests was performed, which allowed for easily visualising testlet-level and broader test-level statistics.

The need for teaching German as second (i.e., German taught in Germany as a foreign language) or foreign language in a university context is growing steadily, both within German-speaking countries and abroad. This requires a means of adequately assessing learners' language skills by means of a placement test, as well as designing lessons and individual learning units according to individual language levels. These lessons, where applicable or necessary, can be offered digitally with automated feedback.

C-Test[editor]

C-tests have long been proven to be very reliable and meaningful as a placement test format at the university level. In order to facilitate digitalisation of language learning and teaching, the C-Test[editor] has now been extended to include German as part of the C-test project at the University of Tübingen, developing linguistic and pedagogical concepts in cooperation with other universities. The C-Test[editor] for German currently enables users to automatically gap the second half of every second word, according to the C-test principle, and to export the gapped text. In development are also options for language-specific characteristics of German, such as the productive occurrence of multi-part compound nouns. Since a clear reconstruction is not to be expected here, an option will be provided to gap the entire compound as a single word or gap the components separately. The C-Test[editor] interface also provides additional information about the generated testlet, such as the type or frequency of the gapped words. Furthermore, the interface provides some basic statistics on the linguistic complexity and readability, such as the Type-Token Ratio of the text. This facilitates the selection of suitable texts for the creation of testlets.

Automated placement testing is also planned for German as a second language. The concepts and format required for this are currently in development.

Language exercises

Finally, an additional platform linked to the C-Test[editor] is being developed in parallel. This platform is planned to host digitalised language exercises for learners of German which will give learners individual feedback (correct/incorrect answers, as well as more detailed hints to guide learners to the correct answers) and provide teachers with the opportunity to choose between different exercise formats, proficiency levels, and language-specific options.

Italian has been implemented in the C-Test[editor] from the beginning, so most of the work lies in creating new features. C-tests for Italian have been tested for many years in both a paper format and electronically at the Foreign Language Center (FSZ) at the University of Tübingen, yielding a large collection of results for comparison. Currently, the C-Test[editor] offers a wide selection of text statistics for Italian, such as an overview of the number of nouns, verbs, repeated gaps, etc. in the text.

C-tests in Italian have also been implemented as tests for general language proficiency assessment at the Department of Romance Languages at the University of Tübingen via the Ilias platform. An automated analysis on the results of the placement tests was performed, which allowed for easily visualising testlet-level and broader test-level statistics.

In teaching Russian as a foreign language, there exists the need for a tool which can automatically generate a valid digital placement test for learners. As the C-test format for Russian is still relatively rare, the conceptual approach for C-tests in Russian was developed as part of the C-test project in collaboration with LEAD and the Department of Slavic Studies. Starting in 2021, digital C-tests offered via the Ilias platform are in use as placement tests for Russian courses at the Department of Slavic Studies, with an automated analysis able to be generated for the results in order to allow for the easy visualisation of testlet-level and broader test-level statistics.

Additionally, there is a currently on-going Russian validation project in which C-tests are being offered to learners and teachers from universities all over Germany in order to collect data on Russian C-tests and further research in this field. This project includes not only C-tests for foreign language learners, but additionally a tailored Cloze test targeted at heritage speakers.

C-Test[editor]

The Russian extension of the C-Test[editor] has been developed in collaboration with LEAD and the Department of Slavic Studies at the University of Tübingen as part of a Master thesis in Computational Linguistics.

Along with the C-test itself, the C-Test[editor] also provides users with some linguistic complexity and readability measures such as various type-token ratios in order to help determine the difficulty level of a text and its appropriateness for foreign language learners and heritage speakers at various proficiency levels. The tool also offers a variety of text statistics, such as gap length, types of gapped words, etc., in order to factor in different teaching and learning needs (e.g., assisting language teachers in quickly and efficiently determining different levels of competence in Russian).

Language exercises

Currently in development for Russian as part of a Master thesis in Computational Linguistics are automatically generated language exercises aimed at heritage speakers.

Spanish has been implemented in the C-Test[editor] from the beginning, so most of the work lies in creating new features. C-tests for Spanish have been tested for many years in both a paper format and electronically at the Foreign Language Center (FSZ) at the University of Tübingen, yielding a large collection of results for comparison. Currently, the C-Test[editor] offers a wide selection of text statistics for Spanish, such as an overview of the number of nouns, verbs, repeated gaps, etc. in the text.

C-tests in Spanish have also been implemented as tests for general language proficiency assessment at the Department of Romance Languages at the University of Tübingen via the Ilias platform. An automated analysis on the results of the placement tests was performed, which allowed for easily visualising testlet-level and broader test-level statistics.