Assessing ChatGPT’s legal reasoning in statutory land consolidation: The case of Cyprus

Abstract

Land consolidation remains a cornerstone of rural development, but its implementation is often hindered by complex statutory frameworks and lengthy procedures. In Cyprus, where land consolidation has historically reduced fragmentation, the process faces new challenges amid declining agricultural importance and evolving policy priorities such as climate change resilience, sustainable development, and urban land readjustment. At the same time, artificial intelligence (AI) and large language models (LLMs) like ChatGPT are increasingly being considered as tools to support legal and planning processes. This study provides the first systematic evaluation of ChatGPT’s capacity to interpret and respond to legal questions derived from Cyprus’s Land Consolidation Law. Using a corpus of 100 questions across four levels of legal complexity, responses were assessed with a rubric measuring correctness, completeness, clarity, and interpretive depth. The results show strong performance in basic factual and procedural questions (98% accuracy for Type 1), moderate reliability in procedural and hypothetical reasoning (84% and 82% respectively), but significant decline in complex interpretive tasks (55% for Type 4). These findings highlight both the potential and the limitations of LLMs in statutory interpretation: they can provide accessible explanations and procedural guidance but cannot yet replace expert legal reasoning in ambiguous or high-stakes cases. The study contributes to the emerging discourse on AI in land policy and rural development, offering methodological insights for evaluating LLMs in domain-specific legal contexts and outlining implications for their responsible integration into planning, cadastral, and governance workflows.