S103 REFERENCE GUIDE

Table of Contents

Nothing is perfect, so it is possible that there are still some errors in this document. If you find a potential error, please email Mr Nicola Roberto ZEMA (nicola.zema@lisn.upsaclay.fr) under the heading:

Disclaimer

S103 --- Documentation.

1. Introduction

Welcome to S103.

1.1. Work organization

The students will work in trinomes.

  • Each group will have a reference teacher.
  • Each group will have a group responsible.
  • Each student have to choose the group companions independently
    • To select the group, it is necessary to use the group selector applet on moodle (on the course page).
    • Befor the first session, students left out of the groups, will be put into groups randomly.

    • The group responsible needs to be "elected". By the end of the S.A.E. all groups should deliver a written report of what they have done.

      In absence of a report by the end of the S.A.E. the mark "0" could be applied to the group components.

      If you finish the project before the end of the semester, that's it. It is not necessary to come to the IUT but you will have to notify the teacher that you have completed the project and the teacher will have to verify that you did it.

1.2. Teachers

The teachers involved in S103 for 2024 will be:

2. Materials given to each group

For the S.A.E. each and every group will use:

  • 1 Raspberry Pi 400
  • 1 SD card
  • 1 an SD card reader/adapter
  • 1 Ethernet-to-USB adapter
  • 1 UTP Patch (short Ethernet cable)
  • 1 HDMI-to-miniHDMI cable (to connect the Raspberry to a monitor)

  • 1 USBa-to-USBc cable (to power the Raspberry)

    At the beginning of each session the teacher will give each group the material. At the end of the each session the students have to stove correctly the material. The students will keep ONLY the SD Card they were given and stove the other equipment with the help of the teacher.

    The materials need to be checked before and after each S.A.E. session by each student group AND the teacher. Eventual malfunctioning has to be reported as for any other in the computing infrastructure of the Orsay IUT.

2.1. Halls

Each SAE session will be in a hall at the IUT.

The SAE puts an emphasis of individual work and independent though. The teacher is there only to give hints and handle extreme cases.

All groups will cycle between:

  • Hall I 224-226 (Salle X)
  • Hall I 225-227 (Salle H)
  • Hall I 219 (Salle S)
  • Hall I 217 (Salle T)

2.2. Material Localization

You are allowed to keep with each group only the SD card with the saved work.

The rest of the material has to be left in the halls and safely stoved at the end of each session.

2.3. Internet and Proxy

It is supposed that the main helping hand for any SysOp is, as of 2024, the biggest manual of it all: Internet.

  • The students are supposed and encouraged to use the information available on the Internet to complete their objective (install and configure a DBMS on a Raspberry Pi 400).

    In fact, being able to read manuals available on the Internet and find out how to do things on-the-fly is one of the most important "competences" that a SysOp need to have.

    The students will be left to find out by themselves how to "do things" by scavenging developer's forums and social platform.

    As of 2024 it is not even necessary to "ask" questions to people about the task of doing SysOp a Raspberry Pi. The information is all already there.

    The use of ChatGPT and other LLMs is ENCOURAGED as they are the best manual of it all. As long as they work they are the best assistant of them all. Just remember that they could be wrong and "hallucinating" (https://machinelearningmastery.com/a-gentle-introduction-to-hallucinations-in-large-language-models/).

3. Objectives

The final objective of this S.A.E. is to have the ability and capabilities to do SysOp. In order to verify these abilities and capabilities, let's start from something you know from R105.

3.1. References

  • Remember that there are no accents by default.

3.1.1. ACTICAMPING

NumCamping NumActivite PrixActivite
1 101 20.50
1 102 15.75
2 101 18.00

3.1.2. ACTIVITE

NumActivite NomActivite TypeActivite
101 Randonnee pedestre Plein air
102 Escalade Aventure
103 Yoga Bien-être

3.1.3. CAMPING

NumCamping NomCamping AddrCamping TelCamping DateOuv DateFerm NbEtoiles QualiteFrance
1 Le Paradis 123 Rue de la Foret 01 23 45 67 89 2023-05-01 2023-10-31 4 Excellente
2 Belle Nature 456 Avenue des Montagnes 98 76 54 32 10 2023-06-15 2023-09-30 3 Bonne

The final objective of this S.A.E. is to make these tables (with fake data) available on a DBMS on your group's Raspberry. You should install a DBMS and configure it on a bare metal Raspberry PI 400 and document all the steps necessary to reproduce the process.

By being able to install and configure the database on the Raspberry Pi 400, starting from an empty SD card, validates that the student is able to "set up and configure a digital work station".

3.2. Sub-objectives

To get to the final result, you have to pass through these steps.

  1. Install an Operating System on the Raspberry
  2. Do SysOP on the Raspberry (user management and such)
  3. Install a DBMS on the Raspberry
  4. Configure the database on the Raspberry

3.2.1. Evaluation Conditions

Your Installation will be verified and you evaluated (individually and as a group) during your last session. There will be, the last session:

  • an automatic verification of your work on the raspberry, for evaluating group work, (60% Eval)
  • A brief QCM, for evaluating INDIVIDUAL work (20% Eval)
  • A 5-pages max report in ENGLISH of your work (20% Eval) for the English part.
  1. Verification
    • The verification is to be done starting from a not powered on Raspberry.
      • This means that the server should be configured to start by itself when the raspberry is powered on (see the SysOp part).

3.2.2. Evaluation Method

As your Raspberry is connected to an IUT PC, a teacher can access it remotely and run a verification script that checks the presence of components automatically and print a mark. Well, I am giving to you the script we the teachers are going to run to verify the Raspberry in order for you to prepare well.

  • WARNING
    • This script is run by the professor, with root access, on the IUT computer where the raspberry is connected.
    • to automatically get your group names, it ALSO try to create a ssh connection for a specific user in the raspberry and searches for a file.
    • The script is subject to improvements.
    • From the teacher's side, we have knowledge of all the information as
      • the raspberry IP is fixed
      • we access the student's PC from our PC automatically via SSH (the script is run on the the IUT computer where the raspberry is connected.)
"""
Script for evaluating the configuration of a Raspberry Pi in a student lab.

The script connects to a Raspberry Pi with a fixed IP address (10.42.0.2),
evaluates the SSH connection, retrieves content from students.txt, and performs
checks on the MySQL database structure and content.

Author: NRZ
Date: 04-dec-2024
"""
import subprocess
import shlex
import mysql.connector


def compare_table_structure(table_name, actual_structure):
    """
    Compare the structure of a table with the desired structure.

    Args:
    - table_name (str): The name of the table.
    - actual_structure (list): The actual structure of the table.

    Returns:
    - bool: True if the table structure matches the desired structure, False otherwise.
    """
    desired_columns = set(desired_structure[table_name])
    actual_columns = set(column[0] for column in actual_structure)
    return desired_columns == actual_columns


def print_table_content(cursor, table_name):
    """
    Print the content of a table.

    Args:
    - cursor: The MySQL cursor.
    - table_name (str): The name of the table.
    """
    cursor.execute(f"SELECT * FROM {table_name}")
    rows = cursor.fetchall()

    print(f"\nTable {table_name} content:")
    for row in rows:
        print(row)


def evaluate_ping(ip):
    """
    Evaluate the response to a ping command.

    Args:
    - ip (str): The IP address to ping.

    Returns:
    - bool: True if the ping is successful, False otherwise.
    """
    ping_command = ["ping", "-c", "1", ip]
    try:
        subprocess.run(ping_command, check=True)
        return True
    except subprocess.CalledProcessError:
        return False


def evaluate_ssh(ip):
    """
    Evaluate SSH connection to a Raspberry Pi and retrieve content from ~/students.txt of the user student.

    Args:
    - ip (str): The IP address of the Raspberry Pi.

    Returns:
    - Tuple[bool, Optional[str]]: A tuple containing a boolean indicating SSH success and the content of students.txt (if available).
    """
    password = 'pwdstudent'
    ssh_command = f"sshpass -p {shlex.quote(password)} ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null student@{ip} 'cat ~/students.txt'"
    try:
        ssh_output = subprocess.check_output(ssh_command, shell=True, text=True)
        return True, ssh_output.strip()
    except subprocess.CalledProcessError:
        return False, None


# Configuration
db_config = {
    'user': 'prof',
    'password': 'pwdprof',
    'host': '10.42.0.2',
    'database': 'CAMPING',
}

# Desired structure of tables
desired_structure = {
    'ACTICAMPING': ['NumCamping', 'NumActivite', 'PrixActivite'],
    'ACTIVITE': ['NumActivite', 'NomActivite', 'TypeActivite'],
    'CAMPING': ['NumCamping', 'NomCamping', 'AddrCamping', 'TelCamping', 'DateOuv', 'DateFerm', 'NbEtoiles', 'QualiteFrance'],
}


try:
    # Ping the IP
    ping_success = evaluate_ping(db_config['host'])

    if not ping_success:
        print(f"\nRaspberry at {db_config['host']}: Ping failed. Exiting and giving 0 points.")
        exit(0)
    else:
        # SSH Connection and evaluation
        ssh_success, ssh_content = evaluate_ssh(db_config['host'])

        if not ssh_success:
            print(f"\nRaspberry at {db_config['host']}: SSH connection failed. Exiting and giving 5 points.")
            exit(5)
        elif ssh_content:
            print(f"\nRaspberry at {db_config['host']}: SSH connection successful. Content of students.txt:\n{ssh_content}")
        else:
            print(f"\nRaspberry at {db_config['host']}: students.txt not found. Exiting and giving 7 points.")
            exit(7)

        # Connect to the Raspberry Pi
        with mysql.connector.connect(**db_config) as conn, conn.cursor() as cursor:
            # Try to connect to the database
            cursor.execute("SELECT 1")

            # Evaluate database connection
            db_success = cursor.fetchone()

            if not db_success:
                print(f"Raspberry at {db_config['host']}: Database connection failed. Exiting and giving 11 points.")
                exit(11)
            else:
                # Loop through tables and compare structure
                tables_correct = True
                for table_name in desired_structure:
                    cursor.execute(f"SHOW COLUMNS FROM {table_name}")
                    actual_structure = cursor.fetchall()
                    print_table_content(cursor, table_name)
                    if not compare_table_structure(table_name, actual_structure):
                        tables_correct = False
                        print(f"Raspberry at {db_config['host']}: Attention - Table {table_name} structure does not match!")

                if not tables_correct:
                    print(f"Raspberry at {db_config['host']}: Database tables are not correct. Exiting and giving 16 points.")
                    exit(16)
                else:
                    # If everything is fine
                    print(f"\nRaspberry at {db_config['host']}: All checks passed. Giving 20 points.")
                    exit(20)
                    # print_table_content(cursor, 'ACTICAMPING')  # Change table name as needed

except subprocess.CalledProcessError:
    print(f"\nRaspberry at {db_config['host']}: Ping failed. Exiting and giving 0 points.")
except mysql.connector.Error as err:
    print(f"Error connecting to Raspberry at {db_config['host']}: {err}")
except Exception as e:
    print(f"An unexpected error occurred: {e}")

The script:

  • checks for ping (5pts)
  • checks for ssh (+2 pts)
    • checks for file in specific place with specific user (+4 pts)
  • checks for database (+5 pts)
  • checks for compliant tables (+4 pts)

3.3. Recommendations

However, to run a DBMS on a Raspberry, it is necessary that an Operating System (OS) is also running on the latter.

  • A raspberry and supporting material will be given to each group.
  • The raspberry itself does not contain any software or OS at all.
  • Before installing the DBMS, the students need to install and configure an operating system for their Raspberry.
    • Even the SD card that will be given to each group will be empty. The students will need to "flash" it with an image of the operating system.

3.3.1. Choice of Operating System

The Pedagogical Equipe suggest to focus on two possibilities for an operating system:

Raspberry Pi OS
Recommended option for anyone. Copious documentation available on the internet.
Arch Linux ARM
Recommended for these that have already some experience with installing operating systems and want a challenge.

3.3.2. Choice of DBMS

  • I did the tests with mariadb.

3.3.3. Extra recommendations

  • No one stops you from installing a VM somewhere with a debian-based linux to do tests from home.
  • DO NOT CONNECT TO EDUROAM OR EDUSPOT VIA WIFI, USE THE CABLE. IF IT DOES NOT WORK, ASK THE TEACHER

4. Expected Results

Each group needs to install first Raspberry Pi OS and then the database on its Raspberry Pi 400

4.1. Translation to practical evaluation

In order to be evaluated, each group needs to:

  1. Produce the proof that they have installed a DBMS on their Raspberry
    • The proof will be a working database installation
    • The teacher will verify that the program is working by running the script.
  2. Verify that each student inside each group has grasped the basic competences envisioned by the SAE by responding to a QCM.
  3. Document the process
    • Deliver a written report about the installation process.
    • In absence of a report deliverance by the end of the S.A.E. the mark "0" could be applied to the group components.
    • Deliver it inside the moodle activity.

4.2. The Report

The report is to be given as a 5-pages maximum PDF Document. Write it using your favorite document editor and then export it to PDF.

AVOID THE USE OF SCREENSHOT AND INSTEAD LEARN HOW TO IMPORT SYNTAX-HIGHLIGHTED-CODE IN YOUR DOCUMENT

4.2.1. English part

Originally the report was 15-pages with 5 pages in english (total: 20).

However, since there is not enough time to correct all your reports in the time frame that starts the day before your last session and the last day of the semester, a single 5-pages report has to be done entirely in English.

4.3. The QCM

It will test the individual knowledge.

It will be in English

It will be on Moodle

5. Material

5.1. Raspberry Pi

Raspberry Pi (/paI/) is a series of small single-board computers (SBCs) developed in the United Kingdom by the Raspberry Pi Foundation in association with Broadcom. The Raspberry Pi project originally leaned towards the promotion of teaching basic computer science in schools and in developing countries. The original model became more popular than anticipated, selling outside its target market for uses such as robotics. It is widely used in many areas, such as for weather monitoring, because of its low cost, modularity, and open design. It is typically used by computer and electronic hobbyists, due to its adoption of HDMI and USB devices. After the release of the second board type, the Raspberry Pi Foundation set up a new entity, named Raspberry Pi Trading, and installed Eben Upton as CEO, with the responsibility of developing technology. The Foundation was rededicated as an educational charity for promoting the teaching of basic computer science in schools and developing countries.

As of May 2021, more than forty million Raspberry boards have been sold. Most Pis are made in a Sony factory in Pencoed, Wales, while others are made in China and Japan.

There are three series of Raspberry Pi, and several generations of each have been released. Raspberry Pi SBCs feature a Broadcom system on a chip (SoC) with an integrated ARM-compatible central processing unit (CPU) and on-chip graphics processing unit (GPU), while Raspberry Pi Pico has a RP2040 system on chip with an integrated ARM-compatible central processing unit (CPU).

5.1.1. Model Reference Table

Raspberry Pi 400 is a Raspberry Pi 4 with integrated hardware.

In Table 1 it is possible to find a list of recent Raspberry Pi models.

Table 1: Raspberry Pi models as of 2021.
Family Model SoC Memory Form Factor Ethernet Wireless GPIO Released Discontinued
Raspberry Pi B BCM2835 256 MB Standard[a] Yes No 26-pin 2012 Yes
  A       No     2013 No
  B+   512 MB   Yes   40-pin 2014 No
  A+     Compact[b] No     2014 No
Raspberry Pi 2 B BCM2836/7 1 GB Standard[a] Yes No   2015 No
Raspberry Pi Zero Zero BCM2835 512 MB Ultra-Compact[c] No No   2015 No
  W/WH         Yes   2017 No
  2 W BCM2710A1       Yes   2021 No
Raspberry Pi 3 B BCM2837A0/B0 1 GB Standard[a] Yes Yes   2016 No
  A+ BCM2837B0 512 MB Compact[b] No Yes (dual band)   2018 No
  B+   1 GB Standard[a] Yes (Gigabit Ethernet)     2018 No
Raspberry Pi 4 B BCM2711 1 GB Standard[a] Yes (Gigabit Ethernet) Yes (dual band)   2019 March 2020
      2 GB           No
      4 GB            
      8 GB         2020  
  400   4 GB Keyboard          
Raspberry Pi Pico N/A RP2040 264 KB Pico (21 mm × 51 mm) No No 26-pin 2021 ?

5.1.2. Raspberry Pi 400

RaspberryPi_Model_4B.png

Figure 1: Raspberry Pi Card

Raspberry_Pi_400_.jpg

Figure 2: Raspberry Pi 400

Raspberry Pi 400 (Figure 1) was released in November 2020. It features a custom board that is derived from the existing Raspberry Pi 4, specifically remodeled with a keyboard attached (Figure 2). The case was derived from that of the Raspberry Pi Keyboard. A robust cooling solution (i.e. a broad metal plate) similar to the one found in Commodore 64 allows the Raspberry Pi 400's Broadcom BCM2711C0 processor to be clocked at 1.8 GHz, which is slightly higher than the Raspberry Pi 4 it's based on. The keyboard-computer features 4 GB of LPDDR4 RAM.

5.2. Operating Systems

The Raspberry Pi Foundation provides Raspberry Pi OS (formerly called Raspberry Pi OS), a Debian-based (32-bit) Linux distribution for download, as well as third-party Ubuntu, Windows 10 IoT Core, RISC OS, LibreELEC (specialized media centre distribution) and specialized distributions for the Kodi media center and classroom management. It promotes Python and Scratch as the main programming languages, with support for many other languages. The default firmware is closed source, while unofficial open source is available. Many other operating systems can also run on the Raspberry Pi. The formally verified microkernel seL4 is also supported. There are several ways of installing multiple operating systems on one SD card.

5.2.1. Raspberry Pi OS

Raspberry Pi OS (formerly Raspbian) is a Debian-based operating system for Raspberry Pi. Since 2015, it has been officially provided by the Raspberry Pi Foundation as the primary operating system for the Raspberry Pi family of compact single-board computers. The first version of Raspbian was created by Mike Thompson and Peter Green as an independent project. The initial build was completed in June 2012.

Raspberry Pi OS is highly optimized for the Raspberry Pi line of compact single-board computers with ARM CPUs. It runs on every Raspberry Pi except the Pico microcontroller. Raspberry Pi OS uses a modified LXDE as its desktop environment with the Openbox stacking window manager, along with a unique theme. The distribution is shipped with a copy of the algebra program Wolfram Mathematica and a version of Minecraft called Minecraft: Pi Edition, as well as a lightweight version of the Chromium web browser.

  1. Description:

    Raspberry Pi OS looks similar to many common desktops, such as macOS, Microsoft Windows, and is most similar to LXDE. The menu bar is positioned at the top and contains an application menu and shortcuts to Terminal, Chromium, and File Manager. On the right is a Bluetooth menu, a Wi-Fi menu, volume control, and a digital clock.

    Packages can be installed via APT, the Recommended Software app, and by using the Add/Remove Software tool, a GUI wrapper for APT.

    PCManFM is a file browser allowing quick access to all areas of the computer, and was redesigned in the first Raspberry Pi OS Buster release (2019-06-20).

    Raspberry Pi OS originally used Epiphany as the web browser, but switched to Chromium with the launch of its redesigned desktop.

    Raspberry Pi OS comes with many beginner IDEs, such as Thonny Python IDE, Mu Editor, and Greenfoot. It also ships with educational software like Scratch and Bookshelf.

  2. References:
    Official Web site
    https://www.raspberrypi.com/software/operating-systems/
    Direct link to download page
    https://www.raspberrypi.com/software/
    Basic Usage
    https://www.raspberrypi.com/documentation/computers/os.html
    General Documentation
    https://www.raspberrypi.com/documentation/

5.2.2. Arch Linux ARM

For those who wants a challenge it is possible to choose to install Arch Linux for ARM instead of Raspberry Pi OS.

The support for Raspberry Pi 400 has not yet tested. Use Arch Linux ARM at your own risk.

Arch Linux ARM is a distribution of Linux for ARM computers. It provides targeted kernel and software support for soft-float ARMv5te, hard-float ARMv6 and ARMv7, and ARMv8 AArch64 instruction sets on a variety of consumer devices and development platforms. Raspberry Pi 3 is a ARMv8. Up to October 2021, Raspberry Pi 400 was not supported by the AArch64 kernel but now, with kernel 5.15, it should work out of the box. Our collaboration with Arch Linux brings users the best platform, the newest packages, and installation support.

Kevin Mihelich is currently Arch Linux ARM's primary developer. Arch Linux ARM is community-developed, with software development and user support provided fully by volunteer effort and donations. Also, unlike other community-supported operating systems such as Ubuntu, Arch Linux ARM has a relatively small user base, making user participation in development especially important.

Arch Linux ARM has a rolling release cycle, i.e. new software is packaged as it is released. This "bleeding edge" release cycle of small, frequent package updates differs from release cycles of Linux distributions such as Debian, which focus on large, scheduled releases of packages proven to be stable.

Arch Linux ARM can run on any device that supports ARMv5te, ARMv6h, ARMv7 or ARMv8 instruction sets. However, support can be limited in some cases. For example, although the Raspberry Pi 4B offers the 64-bit ARMv8 instruction set, on this processor Arch Linux ARM uses 32-bit ARMv7 as the 4B is not yet fully supported by the Linux kernel, however an experimental ARMv8 version is provided, although full hardware support is not guaranteed.

Up to October 2021, Raspberry Pi 400 was not supported by the AArch64 kernel but now, with kernel 5.15, it should work out of the box.

6. Physical Installation

Physical Installation Guide
https://www.raspberrypi.com/documentation/computers/getting-started.html

In order to make the Raspberry Pi power up and get an internet connection, it is necessary to attach it to a power supply and to an Ethernet Cable.

To start up, you cannot connect the Raspberry Pi directly to the power grid and network. You need to attach it first to a IUT computer, authenticated via a username/password pair.

You, as a group, were given an USB cable, a USB network adapter and a short Ethernet cable. You need to:

  • Power up the PC you use as a group and select GNU/Linux Debian at the boot menu.
  • Login to the machine
  • Attach the USB network adapter to a free USB port on the fixed PC
  • Attach the two ends of the network cable one to the network adapter and one to the Raspberry Pi
  • Attach the HDMI cable to the monitor
  • Switch the monitor to display the external input
  • Attach the USB power supply between the Raspberry Pi and the fixed PC. If you did not install any Operating System on the Raspberry, you should not see anything on the screen. Otherwise, if the SD Card contains an operating system, then the Raspberry Pi should power up and load.

6.1. Things to take into account for the IUT PCs

To start up and connect the Raspberry Pi to the internet, you need to connect it first to a IUT PC and then use it.

There are, however, some quirks to take into account while operating the raspberry as you don't connect directly to the internet but through the IUT infrastructure.

There is a set of automatic scripts put in place by the pedagogical team of S103 that will automatically fire up when a network adapter for the Raspberry Pi is detected.

For the moment, you only take into account the following.

  • FIRST BOOT UP THE IUT PC
  • THEN ASK TO OPEN THE INTERNET CONNECTION
  • THEN BOOT UP THE RASPBERRY
  • WATCH OUT FOR TIME AND DATE SETTINGS ON THE RASPBERRY
  • DO NOT CONNECT TO EDUROAM OR EDUSPOT VIA WIFI, USE THE CABLE. IF IT DOES NOT WORK, ASK THE TEACHER

6.1.1. Software for flashing the SD card

In all the guides it is written that you need a specific software that needs to be run with privilege escalation for flashing the SD card: rpi-imager

In the IUT PCs we provide you with an already installed version.

You need to run it via sudo.

A password will not be asked for.

There is an image of RaspberrOS on /Pub/S103/

6.2. Things to take into account on the Raspberry Pi

6.2.1. General Internet Connection

After connecting the cable there will be no internet connection.

There is a set of internal scripts, not accessible by non-root users, that enable internet connection in the raspberry and these scripts are linked to the command that opens the internet connection in the salle.

6.2.2. NTP

The Rasberry Pi does not have a BIOS battery so the clock will probably be out of synch.

Normally, there is a NTP server running on the Raspberry but, as there will be no internet immediately, it will not be possible for the Raspberry to get the correct time from outside.

For this reason, some advanced services, like HTTPS, COULD not work out-of-the-box, if the difference between the current time and the last time saved on the Raspberry is too large.

To fix this, you need to check up and set up the time manually on your Raspberry at each start up.

6.3. Raspberry Pi OS

To install Raspberry Pi OS on a raspberry, it is necessary to follow the guide here

7. Installing a DBMS

Please follow the instructions in the link.

Date: \today

Author: Nicola Roberto Zema

Created: 2024-12-04 mer. 10:53

Validate