COLLECTING TEXTUAL DATA FROM WEBSITES FOR LOW-RESOURCE LANGUAGES - Uteuliev, Nietbay Uteulievich; Abdalieva, Gulistan Rafikovna; Kalmuratov, Bekbos Kushkinbaevich; Asenbaeva, Dalloris Adilbayevna (SCIENCE and SOCIETY, 2026)
This article focuses on the automatic collection of textual data from websites for low-resource languages, using the Karakalpak language as a case study. The study developed a methodology for collecting article texts from kknews.uz, yuz.uz, and joqargikenes.uz using Python, Requests, and BeautifulSoup. As a result, 1 386 navigation pages were analyzed, and a primary raw text database containing 18 784 articles and 6 090 553 words was formed. The collected raw corpus may serve as an initial resource for further preprocessing and for future development of NLP systems, machine translation tools, and Transformer-based models for the Karakalpak language.