AUTO-Update of openvpn configuration

Written by pmd - - no comments

It happens that NordVPN openvpn configuration files don't work anymore after a while.

I made a bash script to update the configuration easy and fast:

#!/bin/bash
# Mise a jour de la configuration NordVPN
# Dossier temporaire
DOSSIER_TEMP="/home/pi/NordVPNautoUpdate"
#Dossier des configurations openvpn
DOSSIER_OPENVPN="/etc/openvpn"
CONF_OPENVPN="server.conf"
AUTOLOGIN_OPENVPN="login.txt"
# Pays
PAYS="ua"
# TCP ou UDP?
PROTO="udp"
# Archive de configuration NordVPN
ARCHIVE_NORDVPN="https://downloads.nordcdn.com/configs/archives/servers/ovpn.zip"
FICHIER=$(echo -n $ARCHIVE_NORDVPN | awk -F "/" '{printf $NF}')

# On supprime et on crée le dossier temp
rm -rf $DOSSIER_TEMP
mkdir $DOSSIER_TEMP

# On télécharge tous les fichiers de conf NordVPN
wget $ARCHIVE_NORDVPN -P $DOSSIER_TEMP

# On dézip l'archive téléchargée
mkdir $DOSSIER_TEMP/temp
unzip -q "$DOSSIER_TEMP/$FICHIER" -d $DOSSIER_TEMP/temp

# On garde que les conf d'un certain pays
mv $DOSSIER_TEMP/temp/ovpn_$PROTO/$PAYS[0-9]*$PROTO* $DOSSIER_TEMP/
rm -r $DOSSIER_TEMP/temp

# On supprime les conf double-VPN et l'archive zip
NbConf=0
for ConfOpenVPN in $DOSSIER_TEMP/*; do
        if [[ $ConfOpenVPN != */$PAYS[0-9]*$PROTO* ]]; then
                rm $ConfOpenVPN
        else
                # On compte le nombre de conf restantes
                NbConf=$((NbConf+1))
        fi
done

# On choisi une conf au hasard
NbConf=$((1 + RANDOM % $NbConf))
COMPTEUR=0
for ConfOpenVPN in $DOSSIER_TEMP/*; do
        COMPTEUR=$((COMPTEUR+1))
        if [[ $COMPTEUR -eq $NbConf ]]; then
            # On modifie la configuration pour authentification automatique
            sed -i "s@auth-user-pass@auth-user-pass $DOSSIER_OPENVPN/$AUTOLOGIN_OPENVPN@" $ConfOpenVPN
            # On déplace la conf dans le dossier openvpn
            sudo cp -f $ConfOpenVPN $DOSSIER_OPENVPN
            sudo cp -f $ConfOpenVPN $DOSSIER_OPENVPN/$CONF_OPENVPN
                        # On informe
                        echo "Configuration installée :"
                        echo $ConfOpenVPN | awk -F "/" '{printf $NF}'
                        echo ""
        fi
done

# On supprime le dossier temporaire
rm -rf $DOSSIER_TEMP
# On redémarre openvpn avec la nouvelle configuration
sudo service openvpn restart

Then simply execute the script:

bash NordVPNautoUpdate.sh

Unlock standalone Airsonic database

Written by pmd - - no comments

If your airsonic is stucked for some reasons such as Liquibase Update Failed and Waiting for changelog lock...

liquibase.exception.LockException: Could not acquire change log lock. Currently locked by 192.168.254.200 (192.168.254.200) since 18/11/18 16:39
  1. Stop Airsonic for sure :
    sudo service airsonic stop
  2. Read logs /var/airsonic/airsonic.log :
    nano /var/airsonic/airsonic.log
  3. Backup your database folder /var/airsonic/db :
    sudo cp /var/airsonic/db /var/airsonic/db_backup
  4. Edit /var/airsonic/db/airsonic.script :
    sudo nano /var/airsonic/db/airsonic.script
    • remove all lines where you can find DATABASECHANGELOGLOCK by example :
      • CREATE MEMORY TABLE DATABASECHANGELOGLOCK.....
      • INSERT INTO DATABASECHANGELOGLOCK VALUES(1,TRUE,'2018-11-18 16:39:56.266000000','192.168.254.200 (192.168.254.200)')
  5. Save file.
  6. Start Airsonic :
    sudo service airsonic start
  7. Check Airsonic status :
    sudo service airsonic status

If you wanna reset your Airsonic music server, just delete the database /var/airsonic/db ...

Source 1, 2, 3.

Update Airsonic on Raspberry Pi 3

Written by pmd - - no comments

How to update a standalone Airsonic

  • Stop the current running Airsonic:
$ sudo systemctl stop airsonic.service
$ cd /var/airsonic/
$ sudo mv airsonic.war airsonic.war.10.1.1.bak
$ sudo wget https://github.com/airsonic/airsonic/releases/download/v10.1.2/airsonic.war --output-document=/var/airsonic/airsonic.war
  • Put good rights to all folder (new airsonic.war is not owned by airsonic), and verify it has been applied:
$ sudo chown -R airsonic:airsonic /var/airsonic
$ ls
total 151M
-rw-r--r--  1 airsonic airsonic 760K Oct 11 19:52 airsonic.log
-rw-r--r--  1 airsonic airsonic  11M Oct  3 00:20 airsonic.log.1
-rw-r--r--  1 airsonic airsonic  978 Oct 11 04:00 airsonic.properties
-rw-r--r--  1 airsonic airsonic  70M Jul 28 22:10 airsonic.war
-rw-r--r--  1 airsonic airsonic  70M Dec 16  2017 airsonic.war.10.1.1.bak
drwxr-xr-x  2 airsonic airsonic 4.0K Oct 11 19:49 db
drwxr-xr-x  2 airsonic airsonic  20K Sep 14 16:08 lastfmcache
drwxr-xr-x  7 airsonic airsonic 4.0K Jul 22 10:36 lucene2
-rw-r--r--  1 airsonic airsonic  741 Oct 11 19:50 rollback.sql
drwxr-xr-x 16 airsonic airsonic 4.0K Oct  4 17:45 thumbs
drwxr-xr-x  2 airsonic airsonic 4.0K Jul 22 10:54 transcode
  • Start updated Airsonic and restart proxy server:
$ sudo systemctl start airsonic.service
$ sudo service nginx restart

Nothing too hard, basically, you need to replace the airsonic.war by the updated one.

Unstable Wifi connection to Raspberry Pi 3

Written by pmd - - no comments

Original post

I am loosing connection to my Raspberry Pi 3 in a very much random way.

I recently replaced Subsonic by Airsonic, and connection to Pi seemed to be much more stable since that exact day. But today, it happened one more time... as I was expecting! Time to troubleshoot has come.

Since I am using 2 tables to root my traffic, one of them is not updated on some interface up/down events. I made a small script called by cron every 5 minutes to understand things and rebooting Pi once a day to get the access back:

#!/usr/bin/env bash

# Script to monitor and restart wireless access point when needed

# Logging
date >> NetworkCheckLog
echo "wap_check_pmd: Checking if network connection is down..." >> NetworkCheckLog
echo "$ ip route show table main" >> NetworkCheckLog
ip route show table main >> NetworkCheckLog
echo "$ ip route show table 42" >> NetworkCheckLog
ip route show table 42 >> NetworkCheckLog

# First make sure we can resolve google, otherwise 'ping -w' would hang
if ! $(host -W5 www.google.com > /dev/null 2>&1); then
        #Make a note in syslog
        logger "wap_check_pmd: Network connection is down..."
        date >> NetworkCheckLog
        ifconfig >> NetworkCheckLog
        echo "wap_check_pmd: Network connection is down..." >> NetworkCheckLog
        exit
fi

Time when it got fucked:

Mon  6 Aug 08:55:01 CEST 2018
wap_check_pmd: Checking if network connection is down...
$ ip route show table main
0.0.0.0/1 via 10.8.7.1 dev tun0
default via 192.168.1.1 dev wlan0 src 192.168.1.200 metric 303
10.8.8.0/24 dev tun0 proto kernel scope link src 10.8.7.12
31.192.112.170 via 192.168.1.1 dev wlan0
128.0.0.0/1 via 10.8.7.1 dev tun0
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.200 metric 303
$ ip route show table 42
default via 192.168.1.1 dev wlan0 src 192.168.1.200 metric 303
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.200 metric 303

Mon  6 Aug 09:00:01 CEST 2018
wap_check_pmd: Checking if network connection is down...
$ ip route show table main
0.0.0.0/1 via 10.8.7.1 dev tun0
default via 192.168.1.1 dev wlan0 src 192.168.1.200 metric 303
10.8.8.0/24 dev tun0 proto kernel scope link src 10.8.7.12
128.0.0.0/1 via 10.8.7.1 dev tun0
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.200 metric 303
$ ip route show table 42
Mon  6 Aug 09:00:15 CEST 2018
wap_check_pmd: Network connection is down...

I then checked /var/log/syslog:

Aug  6 08:59:26 raspberrypi dhcpcd[382]: wlan0: carrier lost
[...]
Aug  6 08:59:26 raspberrypi dhcpcd[382]: wlan0: deleting default route via 192.168.1.1
Aug  6 08:59:26 raspberrypi dhcpcd[382]: wlan0: deleting route to 192.168.1.0/24
[...]
Aug  6 08:59:51 raspberrypi dhcpcd[382]: wlan0: carrier acquired

After Googling, I found an interesting article. Which provide interesting tips.

  1. Use this command to read the current power saving mode of your Pi:
    $ sudo iw wlan0 get power_save
    Power save: on
  2. And this one to turn power_save off:
    $ sudo iw wlan0 set power_save off
  3. To make this permanent add the following line to /etc/rc.local:
    /sbin/iw dev wlan0 set power_save off

We will see if it helps :)

Update 27/10/2018

I choose to turn the power_save off as it originally was. But I came up with this script:

#!/bin/bash
#/home/pi/wap_check
# Script to monitor and restart wireless access point when needed
maxPloss=10 #Maximum percent packet loss before a restart

# Logging
# echo "$(date +"%d-%m-%y %H:%M:%S ") wap_check_pmd: Checking if network connection is down..." >> /home/pi/NetworkCheckLog

# If table 42 is empty, we lost raspberry
TABLE_42=$(ip route show table 42)
if [[ -z "${TABLE_42}" ]]; then
  # Loging
  echo  "$(date +"%d-%m-%y %H:%M:%S ") wap_check_pmd: Table 42 is empty! Trying to populate it:" >> /home/pi/NetworkCheckLog
  logger "wap_check_pmd: Table 42 is empty! Trying to populate it."
  # Filling up the table 42
  ip route show table main | grep -v tun0 | while read LINE; do sudo ip route add $LINE table 42; done
  sleep 1
  ip route show table 42 >> /home/pi/NetworkCheckLog
fi

# IP cannot be the same, debian-transmission can go only through tun0
IP_VPN=$(sudo -u debian-transmission wget -qO- ifconfig.co)
IP_PUBLIC=$(wget -qO- ifconfig.co)
if [[ "${IP_VPN}" = "${IP_PUBLIC}" ]]; then
  echo "$(date +"%d-%m-%y %H:%M:%S ") wap_check_pmd: Same IP for debian-transmission and pi: $(echo $IP_VPN) and $(echo $IP_PUBLIC)" >> /home/pi/NetworkCheckLog
  logger "wap_check_pmd: Same IP for debian-transmission and pi..."
  if [[ -n "${IP_PUBLIC}"  ]]; then
    echo "$(date +"%d-%m-%y %H:%M:%S ") wap_check_pmd: Rebooting. [1]" >> /home/pi/NetworkCheckLog
        logger "wap_check_pmd: Rebooting. [1]"
    sleep 1
    sudo /sbin/reboot
        exit
  fi
fi

# Make sure we can resolve google, otherwise 'ping -w' would hang
if ! $(host -W5 www.google.com > /dev/null 2>&1); then
    # Make a note in syslog
    logger "wap_check_pmd: Network connection is down..."
        echo "$(date +"%d-%m-%y %H:%M:%S ") wap_check_pmd: Rebooting. [2]" >> /home/pi/NetworkCheckLog
        logger "wap_check_pmd: Rebooting. [2]"
        # Restart device
        sleep 1
        sudo /sbin/reboot
    exit
fi

# Initialize to a value that would force a restart
# (just in case ping gives an error and ploss doesn't get set)
ploss=101
# now ping google for 10 seconds and count packet loss
ploss=$(ping -q -w10 www.google.com | grep -o "[0-9]*%" | tr -d %) > /dev/null 2>&1
if [ "$ploss" -gt "$maxPloss" ]; then
    logger "wap_check_pmd: Packet loss ($ploss%) exceeded $maxPloss"
    echo "$(date +"%d-%m-%y %H:%M:%S ") wap_check_pmd: Packet loss ($ploss%) exceeded $maxPloss" >> /home/pi/NetworkCheckLog
    # restart_networking
fi

I run this script every 5 minutes, line in my crontab file:

*/5 * * * * /bin/bash /home/pi/NetworkCheck.sh

Log file looks like this:

22-09-18 22:25:02  wap_check_pmd: Table 42 is empty! Trying to populate it:
22-09-18 22:26:55  wap_check_pmd: Same IP for debian-transmission and pi:  and
22-09-18 22:27:09  wap_check_pmd: Rebooting. [2]
24-09-18 23:20:16  wap_check_pmd: Rebooting. [2]
25-09-18 22:26:10  wap_check_pmd: Packet loss (30%) exceeded 10
25-09-18 23:11:07  wap_check_pmd: Packet loss (16%) exceeded 10
04-10-18 02:55:12  wap_check_pmd: Packet loss (70%) exceeded 10
04-10-18 15:50:01  wap_check_pmd: Table 42 is empty! Trying to populate it:
default via 192.168.1.1 dev wlan0 src 192.168.1.200 metric 303
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.200 metric 303
04-10-18 19:20:12  wap_check_pmd: Packet loss (20%) exceeded 10
04-10-18 20:40:27  wap_check_pmd: Packet loss (30%) exceeded 10
04-10-18 20:55:21  wap_check_pmd: Packet loss (16%) exceeded 10
05-10-18 20:20:01  wap_check_pmd: Table 42 is empty! Trying to populate it:
default via 192.168.1.1 dev wlan0 src 192.168.1.200 metric 303
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.200 metric 303
05-10-18 20:52:25  wap_check_pmd: Packet loss (30%) exceeded 10
06-10-18 15:15:15  wap_check_pmd: Packet loss (30%) exceeded 10
07-10-18 08:55:02  wap_check_pmd: Same IP for debian-transmission and pi:  and
16-10-18 19:21:53  wap_check_pmd: Same IP for debian-transmission and pi:  and
16-10-18 19:22:07  wap_check_pmd: Rebooting. [2]
27-10-18 11:30:20  wap_check_pmd: Packet loss (20%) exceeded 10

youtube-dl

Written by pmd - - no comments

youtube-dl is a usefull packet to download youtube video through command line. GitHub.

List what to download

First step is to list what youtube video you wanna download and convert to mp3 file:

2 files to put on a web server which are:

  1. liste.txt which must be chmod 777 (index.php will write links in this file)
  2. index.php
<!DOCTYPE html>
<html>
<head>
<title>youtube-dl</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
body {
width: 550px;
font-size: 75%;
}
p {
font-family: "Lucida Console", "Courier New", monospace;
white-space: nowrap
}
li {
font-family: "Lucida Console", "Courier New", monospace;
white-space: nowrap
}
hr {
border: 0;
border-bottom: 1px dashed #CCCCCC;
background: #FFFFFF;
}
input[type=text], select {
width: 80%;
padding: 12px 20px;
margin: 8px 0;
display: inline-block;
border: 1px solid #ccc;
border-radius: 4px;
box-sizing: border-box;
}

input[type=submit] {
width: 80%;
background-color: #4CAF50;
color: white;
padding: 14px 20px;
margin: 8px 0;
border: none;
border-radius: 4px;
cursor: pointer;
}

input[type=submit]:hover {
background-color: #45a049;
}

div {
border-radius: 5px;
background-color: #f2f2f2;
padding: 20px;
overflow:auto;
width: 500px;
}
</style>
</head>
<body>
<span style="white-space: nowrap">
<div>
<p>
<?php
if(isset($_POST['field1']))
{
$data = $_POST['field1'] . "\r\n";
$ret = file_put_contents('liste.txt', $data, FILE_APPEND | LOCK_EX);
if($ret === false) {
echo "There was an error writing this file.";
}
else {
echo "$ret bytes written to file.";
}
}
else {
echo "No post data to process.";
}
?>
</p>
</div>
<hr>

<div>

<p>Format to be respected:</p>
<ol>
<li>« https://www.youtube.com/watch?v=1YRW1QRKTBc »</li>
<li>« https://youtu.be/9cBtJYI6itg »</li>
<li>« https://soundcloud.com/hungry-music/nto-alter-ego »</li>
</ol>

<form action="index.php" method="POST">
<input type="text" id="fname" name="field1" placeholder="https://www.youtube.com/watch?v=1YRW1QRKTBc"><br />
<input type="submit" value="Submit">
</form>
</div>

<hr>

<div>
<p>Next link(s) to be downloaded and saved to mp3 files:</p>
<p>
<?php
echo nl2br(file_get_contents( "liste.txt" ));
?>
</p>
</div>

<hr>

<div>
<p>
Logs (last 200 lines): <br />
<?php
//echo nl2br(file_get_contents( "/path/to/history.txt" ));
echo nl2br(trim(implode("", array_slice(file("/path/to/history.txt"), -200))));
?>
</p>
</div>
</body>
</html>

Sources 1, 2

Secure input form

https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/
http://www.htaccesstools.com/htpasswd-generator/

location ^~ /youtube-dl {
     auth_basic           "Иди на хуй мозгоёб!!!";
     auth_basic_user_file /path/to/.htpasswd;
     location ~ \.php$ {
         include snippets/fastcgi-php.conf;
         fastcgi_pass unix:/run/php/php7.0-fpm.sock;
         }
 }

Restart nginx:

$ sudo service nginx restart

Download script

He depends on:

  • /path/to/liste_en_cours.txt
  • /path/to/history.txt
#!/usr/bin/bash
filename="$1"
liste_en_cours=/path/to/Downloads/youtube_auto/liste_en_cours.txt

cp -f $1 $liste_en_cours
sleep 1
echo -n > $1

echo "  " >> /path/to/Downloads/youtube_auto/history.txt
date >> /path/to/Downloads/youtube_auto/history.txt

while read -r line
do
    name="$(echo $line | sed 's/\r//g')"
    echo -n "$name | " >> /path/to/Downloads/youtube_auto/history.txt
    page="$(wget -O - $name)"
    echo $page | sed -n 's/.*<title>\(.*\)<\/title>.*/\1/ip;T;q' >> /path/to/Downloads/youtube_auto/history.txt

    /usr/local/bin/youtube-dl -f bestaudio $name --extract-audio --audio-format mp3 -o "/path/to/Downloads/youtube_auto/downloads/%(title)s.%(ext)s" | grep ffmpeg | grep mp3 | sed 's/.*\//===> /' | sed 's/mp3.*/mp3/' >> /path/to/Downloads/youtube_auto/history.txt

done < "$liste_en_cours"

mv -f /path/to/Downloads/youtube_auto/downloads/* /path/to/Music/youtube

#All big files goes to folder Mixes
for file in /path/to/Music/youtube/*; do
    SIZE="$(stat --printf="%s" "$file")"
    if (( $SIZE > 25000000 )); then
        mv -f "$file" /path/to/Music/Mixes
    fi
done

Source homemade. Now execute it with cron job by example.

 

Rss feed of the category