Escola Superior de Comunicação, Administração e Turismo
Permanent URI for this community
Browse
Browsing Escola Superior de Comunicação, Administração e Turismo by Author "Adão, Telmo"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
- Empowering deaf-hearing communication: exploring synergies between predictive and generative ai-based strategies towards (portuguese) sign language interpretationPublication . Adão, Telmo; Oliveira, João; Shahrabadi, Somayeh; Jesus, Hugo; Fernandes, Marco Paulo Sampaio; Costa, Angelo; Gonçalves, Martinho Fradeira; Lopez, Miguel A.G.; Peres, Emanuel; Magalhães, Luis G.Communication between Deaf and hearing individuals remains a persistent challenge requiring attention to foster inclusivity. Despite notable efforts in the development of digital solutions for sign language recognition (SLR), several issues persist, such as cross-platform interoperability and strategies for tokenizing signs to enable continuous conversations and coherent sentence construction. To address such issues, this paper proposes a non-invasive Portuguese Sign Language (Língua Gestual Portuguesa or LGP) interpretation system-as-a-service, leveraging skeletal posture sequence inference powered by long-short term memory (LSTM) architectures. To address the scarcity of examples during machine learning (ML) model training, dataset augmentation strategies are explored. Additionally, a buffer-based interaction technique is introduced to facilitate LGP terms tokenization. This technique provides real-time feedback to users, allowing them to gauge the time remaining to complete a sign, which aids in the construction of grammatically coherent sentences based on inferred terms/words. To support human-like conditioning rules for interpretation, a large language model (LLM) service is integrated. Experiments reveal that LSTM-based neural networks, trained with 50 LGP terms and subjected to data augmentation, achieved accuracy levels ranging from 80% to 95.6%. Users unanimously reported a high level of intuition when using the buffer-based interaction strategy for terms/words tokenization. Furthermore, tests with an LLM—specifically ChatGPT—demonstrated promising semantic correlation rates in generated sentences, comparable to expected sentences.
- PROMORE: a procedural modeler of virtual rural environments with artificial dataset generation capabilities for remote sensing contextsPublication . Adão, Telmo; Cerqueira, João; Adão, Miguel; Silva, Nuno; Pascoal, David; Magalhães, Luís G.; Barros, Tiago; Premebida, Cristiano; Nunes, Urbano J.; Peres, Emanuel; Morais, RaulRemote sensing (RS) is a rapidly evolving field that facilitates the study of phenomena on the Earth’s surface. Through various platforms, including satellites, manned aircraft, and remotely piloted aerial vehicles (RPAV), RS has been strategically applied to critical sectors like agriculture and forestry, which are essential for humanity’s sustenance. Key applications include crops classification, yield estimation and livestock monitoring and quantification. In the era of artificial intelligence (AI), the development of deep learning (DL) models for such applications often requires extensive field data collection and labor-intensive image labeling, which are both time-consuming and resource-intensive. To address these challenges, this paper presents Procedural Modeling of Rural Environments (PROMORE), a parameterizable, ontology driven system designed to generate 3D virtual environments encompassing forestry, farmland – mainly focused on vineyards – and village settings. This system also implements functionalities to automate the extraction of training data for deep learning applications in remote sensing, with the declared aim of providing complementary capabilities to data augmentation techniques, encompassing both traditional methods (e.g., flips, rotations, zooming) and advanced approaches such as generative adversarial networks (GANs). By simulating RPAV flights and managing virtual object visibility, PROMORE enables the automatic labeling, delineation, and highlighting of elements of interest (e.g., vine plants, trees, buildings), facilitating the generation of datasets tailored for tasks such as semantic segmentation, and object detection.