Projects

Led the circuit experiment segment, focusing on circuit construction and integrated relay control; Used Raspberry Pi I/O interface for seamless hardware-software interaction, ensuring robust PC communication; Collaborated in the implementation of a digital twin 3D model, enabling user interaction with circuit components via a web interface; facilitated real-time data manipulation and experiment data recording Awarded Second Prize in the South China Division of Industry Integration and Innovation Competition

Jun 25, 2024

Project Overview Global freshwater scarcity is a pressing challenge for sustainable development, with agriculture consuming over 70% of the world’s freshwater resources—much of which is wasted due to inefficient irrigation methods. In parallel, modern lifestyles have increased the demand for remote plant care solutions. To address these challenges, I developed an intelligent, fully automatic flowerpot system that integrates cutting-edge technologies such as micropumps, intelligent sensors, IoT cloud platforms, and AI-powered large language models. This system ensures precise control of plant care, providing the ideal environment for growth while significantly reducing water waste. Why We Do This Project: Tackle Global Water Scarcity: Freshwater demand continues to rise, especially in agriculture, which is the largest consumer. This project aims to create a water-efficient solution to reduce waste and optimize resource use. Enhance Agricultural Sustainability: Traditional irrigation methods are often wasteful and lack precision. By enabling accurate, need-based water and nutrient control, this project promotes more sustainable farming practices. Accommodate Busy Lifestyles: With increasingly hectic schedules and frequent travel, people often find it difficult to care for plants consistently. This automated planter ensures reliable plant maintenance even when users are unavailable. Support Plant Enthusiasts and Beginners: Caring for plants, particularly exotic or sensitive species, can be challenging for inexperienced growers. The system provides smart recommendations and automation to simplify plant care. Drive Technological Advancement: By combining AI, IoT, and sensor technologies, this project demonstrates how cutting-edge innovations can address practical challenges and contribute to the development of smart agriculture. 1. System Architecture Figure 1: System Architecture Diagram The architecture illustrates the system's modular design, highlighting the key components: - **Monitoring System**: Tracks air temperature, humidity, soil moisture, and light intensity using advanced sensors and a Pi-Camera for real-time imaging. - **Display System**: Uses the Alibaba Cloud IoT platform for data processing and real-time display on a web-based dashboard. - **Irrigation System**: Incorporates a peristaltic pump and a micropump to deliver precise watering and fertilization. - **Intelligent System**: Leverages ChatGPT for detecting pests, analyzing plant health, and providing actionable insights for maintenance. --- 2. Hardware Components Figure 2: Hardware Physical Picture The hardware framework is centered on a **Raspberry Pi**, which acts as the control hub. It is equipped with various sensors and modules to facilitate data collection and actuation: Environmental Monitoring: Sensors measure humidity, soil moisture, temperature, and light intensity in real-time. Irrigation Control: A peristaltic pump, managed by a relay module, provides precise water delivery to plants. Fertilization Mechanism: A micropump communicates with the software to regulate fertilizer application accurately. High-Definition Camera: Captures real-time images of plants for visual monitoring and AI analysis. The hardware design ensures precise gardening automation by collecting environmental data and implementing corresponding actions through controlled actuation. 3. Software Side Figure 3: Artificial Intelligence Algorithm Flowchart The system's software layer integrates AI technologies and control logic to process data and interact with users: Natural Language Processing: Through OpenAI’s ChatGPT API, the system enables natural language interaction, allowing users to query plant status and receive care advice in real-time. Visual Recognition: Powered by GPT-4’s image analysis capabilities, the system evaluates plant growth, identifying anomalies and providing actionable feedback. Control Algorithms: The software processes sensor data and coordinates irrigation and fertilization schedules to maintain optimal plant health. This layer bridges hardware functionalities with the user interface, enhancing automation and accessibility. 4. Cloud Platform Figure 4: Cloud Platform Display To enable scalability and remote access, the system integrates **Alibaba Cloud's IoT platform**: Data Management: The IoT platform processes large-scale sensor and image data efficiently. Remote Access: Real-time data transmission via Alibaba Cloud OpenAPI allows users to monitor plant conditions and manage devices from anywhere. Custom Programs: Backend scripts handle data preprocessing and transmission, ensuring seamless communication between devices and the cloud. The cloud platform enhances the system’s user-friendliness and scalability by offering robust device management and connectivity. 5. Web Page The web interface is designed for intuitive interaction, displaying real-time plant data and care insights. It is built using HTML, CSS, and JavaScript and features the following core functionalities: 5.1 Webpage Structure and Features The interface is divided into four sections: Environmental Data: Displays real-time measurements of humidity, soil moisture, temperature, and light intensity, each with corresponding timestamps. Plant Image: Shows a high-definition image of the plant, converted from Base64 to JPEG format, with a timestamp for capture time. Suggestions: Provides actionable plant care advice based on real-time analysis, including recommendations for watering, light exposure, and temperature adjustments. Watering and Fertilizing Needs: Displays the plant’s current requirements on a scale, helping users make informed decisions. 5.2 Data Handling Workflow Backend Integration: Data from the IoT platform is processed using a Flask backend, which: Verifies the device’s credentials (device name, product name, product password). Fetches and formats the sensor data. Converts timestamps from milliseconds to human-readable 24-hour format. Prepares the data for frontend presentation. Dynamic Updates: Using JavaScript, the frontend dynamically updates the interface by: Mapping the backend’s data to HTML elements. Rendering Base64-encoded images into readable formats for user convenience. Figure 5: User Web Page 5.3 User Interface Design The webpage is visually structured for clarity and ease of use: Title and Sections: Prominent headers distinguish key sections like “Environmental Data” and “Suggestions.” Styling and Responsiveness: A CSS-based layout ensures the dashboard is aesthetically pleasing and responsive across devices. Accessibility: The interface supports intuitive navigation, making plant monitoring straightforward for all users. 6. Overall Workflow Data Collection: Sensors and the camera collect real-time environmental data and plant images. Local Processing: The Raspberry Pi processes and transmits data to the cloud using the IoT platform. Cloud Management: Alibaba Cloud’s IoT platform handles data storage, processing, and API requests. Backend Operations: The Flask server retrieves and formats the data for frontend integration. Frontend Display: The web interface dynamically displays the processed data, allowing users to monitor and manage their plants effortlessly. 7. Conclusion This system offers a robust and scalable solution for automated plant care by combining precise hardware, intelligent software, and cloud-enabled connectivity. The integration of a dynamic web interface further ensures an intuitive and accessible user experience, making plant monitoring and maintenance both efficient and user-friendly.

Jun 10, 2024

Spearheaded the control segment of a pioneering caterpillar-inspired robot project, focusing on intricate circuit designs to facilitate multiple crawling modes through joule heating and friction manipulation; Designed and executed control circuits on PCB board with NMOS switches for exact thermal-driven motion, integrating Wi-Fi and Bluetooth for dynamic robot control

Feb 16, 2024

Group Members Students: Hao Xizhe (SUSTech), Sun Quanen (UESTC), Wang Gengnan (BUPT), Hong Shiwei (SCU) Professor: Tan Wee Kek (NUS) Teaching Assistants: Wu Haiyang, Li Xin (NUS) Project Overview The Smart Refrigerator leverages AIoT (Artificial Intelligence of Things) principles to monitor and manage refrigerator contents. With this system, users can track food freshness, receive warnings on expired items, and access recommendations for food usage based on fridge contents. Why We Do This Project: In today’s fast-paced world, food waste has become a growing concern, and many households struggle with managing food storage effectively. Our project aims to address these challenges by: Reducing Food Waste: Proactively tracking expiration dates and providing timely alerts to prevent spoilage. Promoting Sustainability: Optimizing food usage with AI-powered recipe suggestions, encouraging users to consume available ingredients efficiently. Enhancing User Convenience: Automating inventory management to save time and effort, making food management smarter and simpler. With these goals, the Smart Refrigerator delivers a blend of convenience, sustainability, and innovation, ensuring smarter food management for households and businesses alike. 0. Poster Download 📄 Download the Project Poster 1. System Roadmap and Core Functionalities The system is built on a Raspberry Pi platform, integrated with sensors and AI-based predictions for real-time monitoring. Core Sensors and Functions: Environmental Monitoring: Tracks temperature, humidity, and door status using sensors. Food Inventory Management: Provides real-time updates on stored food, including weight and freshness predictions. Proactive Alerts: Sends warnings for expired items or abnormal environmental conditions. AI-Driven Suggestions: Recommends recipes based on available ingredients. Figure 1: System architecture with AI and IoT integration 2. Hardware Components The hardware setup is designed to ensure seamless integration and functionality: Sensors collect environmental and food-related data. Data is transmitted to the Raspberry Pi through ESP32 and Micro:bit modules. The Raspberry Pi acts as the central processing hub, analyzing data and interfacing with the AI and web-based components. Key Sensors and Modules: Environmental Sensors: DHT11: Measures temperature and humidity to ensure optimal storage conditions for food items. HC-SR04: Ultrasonic sensor for detecting door status and measuring distances. Passive Buzzer and RGB LED: Provide visual and auditory alerts for warnings, such as when the refrigerator door remains open. Weight Sensor (HX711): Collects weight data of stored food items with precision, enabling effective inventory management. Image Capture and Processing: Pi Camera: Captures images of food items for further processing and classification using AI models. Communication Modules: ESP32: Gathers sensor data (temperature, humidity, door status) and transmits it to the Raspberry Pi via MQTT protocol for real-time monitoring and warnings. Micro:bit Modules: Enable serial and radio communication for capturing, weighing, and managing inventory. 3. Artificial Intelligence Integration The Smart Refrigerator uses a Pre-trained YOLOv8 model to assess food expiration and predict freshness. The Raspberry Pi’s AI capabilities enable real-time predictions with high accuracy, evaluated through: Confusion Matrix and F1-Confidence Curve to track prediction performance. Figure 2: Confusion Matrix for YOLOv8 Model Figure 3: F1-Confidence Curve Showing Model Performance AI Model Workflow The AI integration follows a step-by-step roadmap to achieve accurate and real-time predictions: Step 0: Pre-trained AI Model on PC A YOLOv8 model is pre-trained on a labeled dataset of various food categories to detect and classify food items accurately. Step 1: Pi Camera Capture The Raspberry Pi utilizes the Pi Camera to capture real-time images of food items stored in the refrigerator. Step 2: AI Predictions on Raspberry Pi YOLOv8 is deployed on the Raspberry Pi for on-device inference. It detects food items, classifies them into predefined categories, and predicts the freshness. Step 3: Weight Information Collector The system correlates weight data collected via the HX711 sensor with the visual classification to ensure precise inventory management. Step 4: Update Data on Web Database The classified food data and weight measurements are stored in a cloud-based web database for further analysis and user access. Valid Predictions The YOLOv8 model effectively detects and classifies food items in real-time. The validation predictions show: Successful identification of common food items like apples, bananas, carrots, and fish with high confidence levels. Bounding boxes accurately localize each item in the images, enabling precise tracking. Figure 4: Validated YOLOv8 predictions for some food items. 4. Web Interface and Database Management The system features a web-based interface that allows users to interact with the Smart Refrigerator’s data. Features of the Web Interface Dynamic Dashboard: Displays real-time data on: Environmental conditions (temperature, humidity). Fridge door status (open/closed). Alerts for expiring or expired food items. Powered by HTML, CSS, and JavaScript for a responsive and interactive experience. Food Inventory Management: Lists stored food items along with weight, timestamp, and expiration status. Allows users to: Add or remove items from the inventory. View item-specific storage recommendations (e.g., temperature, shelf life). AI-Powered Recipe Suggestions: Leverages ChatGPT to provide recipe recommendations based on available ingredients. Recipes are dynamically generated using inventory data and a predefined recipe rule set. User-Friendly Alerts: Sends notifications for: Food nearing expiration. Abnormal fridge conditions (e.g., high temperature or humidity). Uses visual cues (color-coded indicators) for quick recognition of issues. Key Pages Index Page: Provides an overview of fridge contents, environmental conditions, and food expiration warnings. Figure 5: Web Interface - Index Page showing fridge contents and expiration alerts. Table List Page: Displays stored food data, including weight, names, and timestamps. Figure 6: Table list displaying detailed storage data. ChatGPT Recommendation Page: Generates recommendations for recipes or meals based on available fridge ingredients. Figure 7: Recommendations based on current ingredients. References Black Dashboard Template: Link Food Library: Link YOLOv8 Documentation: Link Aheleroff S, et al. IoT-enabled smart appliances under Industry 4.0. Advanced Engineering Informatics, 2020. Basa J J A, et al. Smart inventory management system. arXiv, 2019.

Jul 25, 2023