Python on raspberry

Written by pmd - - no comments

Begin

Installer et mettre à jour Python

Python 2 and Python 3 sont préinstallés sur Raspbian, mais au cas ou pour installer python3 :

sudo apt-get install python3

Ecrire un programme en python

Créer un fichier :

nano hello-world.py

Le remplir avec :

#!/usr/bin/python3

print("Hello, World!")

CTRL+X then Y.

Tous les programmes en Python doivent être avec l'extension ".py".

Executer un programme en Python

python3 hello-world.py

Rendre un programme python executable

Pour rendre un fichier ".py" executable :

chmod +x file-name.py

Puis pour l'executer :

./file-name.py

Source

Parse HTML : Beautiful Soup

Installing :

sudo apt-get install python-bs4 python3-bs4

Premier script

#!/usr/bin/python
from bs4 import BeautifulSoup, Comment

print("Hello, World!");

# save as local file 'webpage.html'
import urllib.request;
#urllib.request.urlretrieve ("https://www.xe.com/fr/currencytables/?from=EUR&date=2019-07-11", "webpage.html");

# read entire file and close immediately after block ends
with open('webpage.html', 'r') as f:
    html_doc = f.read()
#print(html_doc);

# on ouvre la page web téléchargée
soup = BeautifulSoup(html_doc, 'lxml')
# on vire tous les commentaires html
for element in soup(text=lambda text: isinstance(text, Comment)):
    element.extract()
# on vire tous les liens
for a in soup.findAll('a'):
    a.replaceWithChildren()

#print(soup.prettify())
all_rates = soup.tbody.find_all("tr")

# Loop over all elements of a list
for element_tr in all_rates:
    print(element_tr)

Installation de modules avec pip

Sous Windows pour installer des modules :

C:\Users\<username>\AppData\Local\Programs\Python\Python37\Scripts>pip3.7.exe install lxml

Sous linux, souvent il y a des paquets déjà prêts. Sinon :

sudo pip install pandas

Faire des choses différentes si script sous Windows ou sous Linux

import platform
if platform.system() == 'Windows':
    #Windows
    work_folder = Path('C:\Users\...')
elif platform.system() == 'Linux':
    #Linux
    work_folder = Path('/home/user/...')

*.py vers *.exe

Installer auto-py-to-exe :

pip install auto-py-to-exe

Lancer le programme depuis le bon dossier :

auto-py-to-exe

Source

 

CSV editor

Written by pmd - - no comments

This script is working under Windows 7 and Windows 10 using Busybox.
To start it I made a shortcut to "busybox.exe sh -l".

To edit CSV for better and easier human reading.

#!/bin/bash
# File to be modified
FILE_IN="$1"
# Output file to be generated
FILE_OUT=$(echo -n "$FILE_IN" | sed 's@.csv@_mod.csv@g')
cp $FILE_IN $FILE_OUT

# delete all empty lines after header (header is 11 lines long)
sed -i '12,${\@^$@d;}' $FILE_OUT

# put back all samples together (delete in-file headers except first one)
sed -i '12,${\@Arming date:@{N;N;d}}' $FILE_OUT

# replace long path by short path of variables
# before: CPU1/CPU_fast//CPU_fast/appli/test/sf_12/do_low_power_tests/n_lpt
# after: CPU1/n_lpt
LINE_TO_MODIFY=$(cat $FILE_OUT | grep "/")
LINE_MODIFIED=$(echo -n "$LINE_TO_MODIFY" | sed -e 's@/[^;]*/@/@g')
echo "Old line to modify:"
echo "$LINE_TO_MODIFY"
echo
echo "New line modified:"
echo "$LINE_MODIFIED"

# remove and replace variable names with short path
sed -i "s@$LINE_TO_MODIFY@$LINE_MODIFIED@g" $FILE_OUT

dhcpcd and dhcpcd-run-hooks on Raspbian

Written by pmd - - no comments

If you are using Raspbian GNU/Linux 9 (stretch), then you are most likely using dhcpcd as your DHCP and DHCPv6 client. How to know your OS version? Here :

$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
  • dhcpcd gets the host information (IP address, routes, etc) from a DHCP server and configures the network interface of the machine on which it is running. Manual of DHCPCD(8).
  • dhcpcd-run-hooks is a DHCP client configuration script. It is used by dhcpcd to run any system and user defined hook scripts. Manual of DHCPCD-RUN-HOOKS(8).

dhcpcd

It seems that to restart all network interfaces this works :

$ sudo systemctl restart dhcpcd.service

To understand what happened, better read dhcpcd logs, here is what we can see:

$ journalctl -u dhcpcd
Dec 21 15:38:04 raspberrypi systemd[1]: Stopping dhcpcd on all interfaces...
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: sending signal TERM to pid 14888
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: waiting for pid 14888 to exit
Dec 21 15:38:04 raspberrypi dhcpcd[14888]: received SIGTERM, stopping
Dec 21 15:38:04 raspberrypi dhcpcd[14888]: eth0: removing interface
Dec 21 15:38:04 raspberrypi dhcpcd[14888]: wlan0: removing interface
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: sending signal TERM to pid 14888
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: waiting for pid 14888 to exit
Dec 21 15:38:05 raspberrypi systemd[1]: Stopped dhcpcd on all interfaces.
Dec 21 15:38:05 raspberrypi systemd[1]: Starting dhcpcd on all interfaces...
Dec 21 15:38:05 raspberrypi dhcpcd[15180]: dev: loaded udev
Dec 21 15:38:05 raspberrypi systemd[1]: Started dhcpcd on all interfaces.
Dec 21 15:38:05 raspberrypi dhcpcd-run-hooks[15203]: wlan0: starting wpa_supplicant
Dec 21 15:38:05 raspberrypi dhcpcd[15185]: eth0: waiting for carrier
Dec 21 15:38:05 raspberrypi dhcpcd[15185]: wlan0: waiting for carrier
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: carrier acquired
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: DUID 00:01:00:01:21:b0:de:63:b8:27:eb:4d:6c:87
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: IAID eb:5d:6d:97
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: adding address fe80::2882:e851:cf0c:9242
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: soliciting an IPv6 router
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: rebinding lease of 192.168.254.200
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: leased 192.168.254.200 for 86400 seconds
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: adding route to 192.168.254.0/24
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: adding default route via 192.168.254.1
Dec 21 15:38:19 raspberrypi dhcpcd[15185]: wlan0: no IPv6 Routers available

Source

dhcpcd-run-hooks

dhcpcd runs /lib/dhcpcd/dhcpcd-run-hooks. This script runs each script found in /lib/dhcpcd/dhcpcd-hooks in a lexical order.
dhcpcd-run-hooks is used by dhcpcd to run any system and user defined hook scripts. System hook scripts are found in /lib/dhcpcd/dhcpcd-hooks.
There is a list of reasons why dhcpcd-run-hooks could be invoked, such as BOUND (= dhcpcd obtained a new lease from a DHCP server = computer has an IP assigned on the interface).
Many variables will then be set, along with any protocol supplied ones, such as :

  • $interface : the name of the interface
  • $reason : reason why dhcpcd-run-hooks was invoked
  • $if_up : true if the interface is up, otherwise false.

To know what is the current IPv4 address after an interface BOUNDed dhcpcd obtained a new lease from a DHCP server.:

ifconfig $interface | grep "inet " | awk '{print $2}'

I created /lib/dhcpcd/dhcpcd-hooks/60-test_by_pmd in order to play with this functionality. Shell script I used in order to check IP after new DHCP lease was obtained (writing output in /home/pi/test.txt in order to see what happened afterward):

if [ "$reason" = "BOUND" ] && [ "$if_up" = "true" ]; then
     echo "$(date +"%d-%m-%y %H:%M:%S ") $reason $interface IP:$if_up DOWN:$if_down" >> /home/pi/test.txt
     ifconfig $interface | grep "inet " | awk '{print $2}' >> /home/pi/test.txt
fi

Sources : both manuals.

AUTO-Update of openvpn configuration

Written by pmd - - no comments

It happens that NordVPN openvpn configuration files don't work anymore after a while.

I made a bash script to update the configuration easy and fast:

#!/bin/bash
# Mise a jour de la configuration NordVPN
# Dossier temporaire
DOSSIER_TEMP="/home/pi/NordVPNautoUpdate"
#Dossier des configurations openvpn
DOSSIER_OPENVPN="/etc/openvpn"
CONF_OPENVPN="server.conf"
AUTOLOGIN_OPENVPN="login.txt"
# Pays
PAYS="fr"
# TCP ou UDP?
PROTO="udp"
# Archive de configuration NordVPN
ARCHIVE_NORDVPN="https://nordvpn.com/api/files/zip"
FICHIER=$(echo -n $ARCHIVE_NORDVPN | awk -F "/" '{printf $NF}')

# On supprime et on crée le dossier temp
rm -rf $DOSSIER_TEMP
mkdir $DOSSIER_TEMP

# On télécharge tous les fichiers de conf NordVPN
wget $ARCHIVE_NORDVPN -P $DOSSIER_TEMP

# On dézip l'archive téléchargée
mkdir $DOSSIER_TEMP/temp
unzip -q "$DOSSIER_TEMP/$FICHIER" -d $DOSSIER_TEMP/temp

# On garde que les conf d'un certain pays
mv $DOSSIER_TEMP/temp/$PAYS[0-9]*$PROTO* $DOSSIER_TEMP/
rm -r $DOSSIER_TEMP/temp

# On supprime les conf double-VPN et l'archive zip
NbConf=0
for ConfOpenVPN in $DOSSIER_TEMP/*; do
        if [[ $ConfOpenVPN != */$PAYS[0-9]*$PROTO* ]]; then
                rm $ConfOpenVPN
        else
                # On compte le nombre de conf restantes
                NbConf=$((NbConf+1))
        fi
done

# On choisi une conf au hasard
NbConf=$((1 + RANDOM % $NbConf))
COMPTEUR=0
for ConfOpenVPN in $DOSSIER_TEMP/*; do
        COMPTEUR=$((COMPTEUR+1))
        if [[ $COMPTEUR -eq $NbConf ]]; then
            # On modifie la configuration pour authentification automatique
            sed -i "s@auth-user-pass@auth-user-pass $DOSSIER_OPENVPN/$AUTOLOGIN_OPENVPN@" $ConfOpenVPN
            # On déplace la conf dans le dossier openvpn
            sudo cp -f $ConfOpenVPN $DOSSIER_OPENVPN
            sudo cp -f $ConfOpenVPN $DOSSIER_OPENVPN/$CONF_OPENVPN
                        # On informe
                        echo "Configuration installée :"
                        echo $ConfOpenVPN | awk -F "/" '{printf $NF}'
                        echo ""
        fi
done

# On supprime le dossier temporaire
rm -rf $DOSSIER_TEMP
# On redémarre openvpn avec la nouvelle configuration
sudo service openvpn restart

Then simply execute the script:

bash NordVPNautoUpdate.sh

Unlock standalone Airsonic database

Written by pmd - - no comments

If your airsonic is stucked for some reasons such as Liquibase Update Failed and Waiting for changelog lock...

liquibase.exception.LockException: Could not acquire change log lock. Currently locked by 192.168.254.200 (192.168.254.200) since 18/11/18 16:39
  1. Stop Airsonic for sure :
    sudo service airsonic stop
  2. Read logs /var/airsonic/airsonic.log :
    nano /var/airsonic/airsonic.log
  3. Backup your database folder /var/airsonic/db :
    sudo cp /var/airsonic/db /var/airsonic/db_backup
  4. Edit /var/airsonic/db/airsonic.script :
    sudo nano /var/airsonic/db/airsonic.script
    • remove all lines where you can find DATABASECHANGELOGLOCK by example :
      • CREATE MEMORY TABLE DATABASECHANGELOGLOCK.....
      • INSERT INTO DATABASECHANGELOGLOCK VALUES(1,TRUE,'2018-11-18 16:39:56.266000000','192.168.254.200 (192.168.254.200)')
  5. Save file.
  6. Start Airsonic :
    sudo service airsonic start
  7. Check Airsonic status :
    sudo service airsonic status

If you wanna reset your Airsonic music server, just delete the database /var/airsonic/db ...

Source 1, 2, 3.

Rss feed of the category