Instruction Extension of RISC-V Processor for Driver Fatigue Detection System and Implementation

Project Partners

Istanbul Technical University
Urmia University
TÜBİTAK

Project Team

Project Summary

Embedded systems are developed specifically to provide cost-effective performance requirements such as speed, accuracy, and reliability. By definition, an embedded system contains application-specific hardware and software components. However, as the target system grows, the workforce and diversity of experience required for system design also increase. Consequently, user control of the system becomes more difficult.

Software support for hardware is essential for making a system easy to use, update when necessary, and operate in coordination with other embedded systems. Many applications can be implemented with inexpensive, application-specific hardware units. However, because of the lack of suitable software development environments, interest often remains focused on expensive, general-purpose solutions designed to support many different applications.

The application areas of embedded systems can be expanded by supporting them with appropriate software and by developing efficient signal-processing and decision-making algorithms specific to these systems. Increasing the number of embedded systems on which researchers and users can easily develop applications enables many solutions to become inexpensive, practical, and responsive to changing conditions.

A complete embedded-system design process must cover all software and hardware components. This requires the combination of hardware, software, and a suitable software-development environment to form the foundation for implementing the application selected in this project. Cost-sensitive application-software design is the final requirement. Therefore, the proposed project is interdisciplinary.

This project includes the design and implementation of a system-on-chip containing an application-specific instruction-set processor and its peripherals. The following stages will be carried out:

  1. Modeling the driver-fatigue monitoring system and verifying the model.
  2. Implementing a RISC-V processor on an FPGA.
  3. Implementing the driver-fatigue monitoring model on the FPGA-based RISC-V processor.
  4. Extending the RISC-V instruction set for driver-fatigue monitoring and implementing the extended processor on the FPGA.
  5. Implementing and testing the driver-fatigue monitoring model using the extended instruction set.
  6. Implementing the instruction-set-extended RISC-V processor as an ASIC.
  7. Implementing and testing the driver-fatigue monitoring model on the RISC-V ASIC using the extended instruction set.

By following these methods, the project aims to reach Technology Readiness Level 5. An application area considered important for both Türkiye and Iran has been selected.

Every year, many traffic accidents occur worldwide because of driver fatigue or loss of concentration, resulting in fatalities and financial losses. Traffic-accident rates in Iran and Türkiye are higher than the world average.

The project will develop a driver-support solution that uses machine-learning and image-processing techniques to address two main issues:

  1. Fatigue measurement.
  2. Concentration measurement.

The resulting solution will be a real-time and inexpensive embedded system designed to assist drivers.

Application-oriented processor extensions will enable real-time and cost-effective implementation of image-processing and machine-learning operations. Designing both the hardware and software required for communication between the processor and its peripherals will allow the system to operate with embedded-system components developed by different hardware manufacturers.

The project will improve the complete design process, from low-level hardware and peripherals to application software and the end-user interface. Through this process, researchers will be able to design and implement systems containing specialized processors that satisfy different application requirements.

The final system-on-chip will also be usable for other applications through software changes. It will therefore be scalable and updateable through application-specific software development.

Project collaboration structure
RISC-V instruction-set extension

Methodology

The processing unit will be designed to access peripherals such as a DMA controller, MIPI CSI-2 camera interface, and USB data connection. To ensure reliable communication between the processor and its peripherals, the processor must be compatible with standard buses such as AXI4.

The AXI bus can support many peripherals through APB bridges. Standard buses provide advantages in terms of hardware availability and software-driver support.

The following fundamental components will first be designed and tested:

  • A software model of the driver-fatigue detection system, including image-processing and machine-learning algorithms.
  • A basic RISC-V implementation on an FPGA.
  • An instruction-set-extended RISC-V implementation on the development board.

Integration 1 — Initial Hardware

Once the initial RISC-V implementation is completed, integration of the processor with the camera and other hardware components on the development board can begin before completion of the software model or instruction-set extension.

This stage will allow obstacles related to initial hardware integration to be detected and resolved. It is also required for collecting real-world data.

Integration 2 — Algorithm with Real Data

Data collected from the hardware implementation will be used to train the driver-fatigue detection model. The model will therefore be integrated with real data obtained from the project setup, allowing its accuracy to be evaluated properly.

Integration 3 — Processor and Software Model

The final model parameters will be used to determine the instruction-set extensions. Close cooperation between the algorithm-design and instruction-extension teams will be required.

This stage will introduce the first implementation of the driver-fatigue detection algorithm on the RISC-V processor.

Integration 4 — Updated Software Model and Processor

After the new instructions have been defined, the driver-fatigue detection algorithm will be revised and rewritten using these instructions. The compiler will also be tested during this integration stage.

The complete software flow, from the high-level algorithm through the compiler to the generated machine code, will be examined.

Final Integration — Complete System

The machine code generated by the compiler will run on the instruction-set-extended RISC-V processor implemented on the FPGA of the development board, together with the cameras and other peripherals.