Wireguard server on Raspberry while being an OpenVPN client

Written by pmd - - no comments

This is follwoing these first notes written a while ago : Wireguard on Raspberry

Context

The Raspberry is already an OpenVPN client : see here.

  • All packets that are not tagged '42' are using main route table (which outputs to OpenVPN tunnel).
  • All packets that are tagged '42' are using route table 42 (which outputs to internet link).

For my use case, all Wireguard packets will need to be routed as specified in table 42.

Wireguard server setup

I used this guide to globally setup wireguard and a few clients (lastest updated using iptables): Installing and Configuring WireGuard on Raspberry Pi OS (September 2021)

The generated configuration was the following:

Server:

$ sudo cat /etc/wireguard/wg0.conf
[Interface]
Address = 192.168.99.1/24
ListenPort = 58280
PrivateKey = gNVxJe7Se842IiOR5GsXeM4sHcacGhPATIdQCgqP8Wa=
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
[Peer]
PublicKey = OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
AllowedIPs = 192.168.99.2/32

Peer 1:

$ cat ~/wg_config/users/client1/client.conf
[Interface]
Address = 192.168.99.2/24
PrivateKey = 6OfJPX1ZQCFu08fTy2uU6JdgUf/qXgzBoTtX/tCYX3a=

[Peer]
PublicKey = b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
AllowedIPs = 192.168.99.1/32, 192.168.1.0/24
Endpoint = adress.ddns.net:58280

Adding other users

⚠️ ⚠️ ⚠️ Be sure to make a copy of your wireguard configuration because it will be overwritten.

$ sudo cp /etc/wireguard/wg0.conf /etc/wireguard/wg0.conf.bak  # make a backup if necessary !
$ sudo wg-quick up wg0                                         # make sure Wireguard is running
$ sudo ./wg_config/user.sh -a another_user                     # creating new user
$ ls ./wg_config/users/another_user/                           # look at all files generated
total 32K
drwxr-xr-x 2 root root 4.0K Sep 21 15:56 .
drwxr-xr-x 6 root root 4.0K Oct  9 18:16 ..
-rw-r--r-- 1 root root  216 Sep 21 15:56 client.all.conf
-rw-r--r-- 1 root root  238 Sep 21 15:56 client.conf
-rw-r--r-- 1 root root  900 Sep 21 15:56 another_user.all.png
-rw-r--r-- 1 root root 1016 Sep 21 15:56 another_user.png
-rw-r--r-- 1 root root   45 Sep 21 15:56 privatekey
-rw-r--r-- 1 root root   45 Sep 21 15:56 publickey
$

IP forwarding

For clients to be able to join each other and access internet, it is necessary to enable IP forwarding:

$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0
$ sudo sysctl -w 'net.ipv4.ip_forward=1'
net.ipv4.ip_forward = 1
$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

Adjustment

Because of the context described above it wasn't enough to have a working link.

I had to add few lines for it to work on server and client sides. See below:

Server:

$ sudo cat /etc/wireguard/wg0.conf
[Interface]
Address = 192.168.99.1/24
ListenPort = 58280
PrivateKey = gNVxJe7Se842IiOR5GsXeM4sHcacGhPATIdQCgqP8Wa=
FwMark = 0x2A # if packet not tagged '42' it will be routed to tun0 interface
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostUp = ip route add 192.168.99.0/24 dev wg0 proto kernel scope link src 192.168.99.1 table 42 # route table update for packet tagged '42'
PostUp = sysctl -w 'net.ipv4.ip_forward=1' # activate IP forwarding
PostUp = ip rule add from 192.168.99.0/24 table 42; ip rule add to 192.168.99.0/24 table 42 # all packet comming/leaving from 192.168.99.0/24 should use table 42

PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
PostDown = sysctl -w 'net.ipv4.ip_forward=0' # disactivate IP forwarding
PostDown = ip rule del from 192.168.99.0/24 table 42; ip rule del to 192.168.99.0/24 table 42

[Peer]
PublicKey = OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
AllowedIPs = 192.168.99.2/32

Sources: FwMark, PostUp.

Peer 1:

$ cat ~/wg_config/users/client1/client.conf
[Interface]
Address = 192.168.99.2/24
PrivateKey = 6OfJPX1ZQCFu08fTy2uU6JdgUf/qXgzBoTtX/tCYX3a=
DNS = 208.67.222.222, 208.67.220.220

[Peer]
PublicKey = b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
AllowedIPs = 0.0.0.0/0
Endpoint = adress.ddns.net:58280

Source DNS, DNS.

Start, monitor and stop wireguard

$ sudo wg-quick up wg0
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 192.168.99.1/24 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
[#] ip route add 192.168.99.0/24 dev wg0 proto kernel scope link src 192.168.99.1 table 42
[#] sysctl -w 'net.ipv4.ip_forward=1'
net.ipv4.ip_forward = 1
[#] ip rule add from 192.168.99.0/24 table 42; ip rule add to 192.168.99.0/24 table 42
$
$ sudo wg
interface: wg0
  public key: b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
  private key: (hidden)
  listening port: 58280
  fwmark: 0x2a

peer: OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=

  endpoint: 96.82.73.111:32378
  allowed ips: 192.168.99.2/32
  latest handshake: 37 seconds ago
  transfer: 425.32 KiB received, 502.92 KiB sent
$
$ sudo wg-quick down wg0
[#] ip link delete dev wg0
[#] iptables -D FORWARD -i wg0 -j ACCEPT; iptables -D FORWARD -o wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
[#] sysctl -w 'net.ipv4.ip_forward=0'
net.ipv4.ip_forward = 0
[#] ip rule del from 192.168.99.0/24 table 42; ip rule del to 192.168.99.0/24 table 42
$

Enabling at startup

Once everything is working you can enable Wireguard at startup by doing:

$ sudo systemctl enable wg-quick@wg0

Boot Raspberry Pi Model 3B V1.2 on SSD

Written by pmd - - no comments

I have a Raspberry Pi Model 3B V1.2. You can know your model by typing in prompt :

$ pinout
,--------------------------------.
| oooooooooooooooooooo J8     +====
| 1ooooooooooooooooooo        | USB
|                             +====
|      Pi Model 3B  V1.2         |
|      +----+                 +====
| |D|  |SoC |                 | USB
| |S|  |    |                 +====
| |I|  +----+                    |
|                   |C|     +======
|                   |S|     |   Net
| pwr        |HDMI| |I||A|  +======
`-| |--------|    |----|V|-------'

In January 2022, the microSD card got corrupted, and Pi was not reliable anymore, after about 3 years of operation.
I bought a new microSD card but as I didn't backup, I had to reinstall all over again. Not a nice feeling.

As of 23/03/2022 I am preparing to configure the Pi to boot on an SSD.

I bought two things so far:

As of 28/12/2022 I red a lot, and nothing worked, until I found something that work pretty nicely.

What I have finally setup

  1. Raspberry Pi is operationnal. Currently running based on the SD Card (FYI 32GB) inserted in the SD Card slot of the Pi.
     
  2. Using SD Card Copier already available in Raspian, I made a clone from the SD Card to the SSD:

     
  3. On your SSD, you should then have 2 partitions, one for boot and one for the OS.
    I formatted another SD Card (FYI 2GB) and I formatted it in FAT32.
    Then, I have made a copy of all the files that were in the boot partition of the SSD to this other 2GB SD Card.
     
  4. I properly switched off the Raspberry Pi.
    I removed the SD Card 32GB from the SD slot that was previously running the system.
    I inserted the SD Card 2GB which contains all the files from the boot partition of the SSD.
    I connected another USB device: a Toshiba HDD 2 TB.
    Everything is powered with the Raspberry Pi 3 Power Supply - 2.5A (Micro USB) by CanaKit. No any USB hub to power the SSD or HDD.

I am a bit concerned about the power. So I will monitor this in the coming days using two commands:

$ vcgencmd get_throttled
throttled=0x0
$ dmesg | grep -iC 3 "under-voltage"

See here for get_throttled signification.

Telegram bot in python

Written by pmd - - no comments

Information from Telegram itself : https://core.telegram.org/bots

YOU: /setjoingroups
BotFather: Choose a bot to change group membership settings.
YOU: @YourBot BotFather: 'Enable' - bot can be added to groups.
'Disable' - block group invitations, the bot can't be added
to groups. Current status is: DISABLED
YOU: Enable
BotFather: Success! The new status is: ENABLED.

Fan control of Raspberry

Written by pmd - - no comments

Hardware

Python3 program

I have red a lot of pages how to control a fan using PWM signal.
Some of them:

 

Finally, it seems my fans don't have much effect on the temperature. About 5°C. So I decided to set an hysteresis:

  • Switch ON fans if temperature > 75°C
  • Switch OFF fans if temperature < 60°C
  • Do nothing if 60°C <= temperature <= 75°C
$ nano gpio_test.py
#!/usr/bin/python3
# -*-coding:Utf-8 -*

import os
from gpiozero import LED
from time import sleep
import RPi.GPIO as GPIO

fanPin = 2
testMode = False

def getCPUtemperature():
    res = os.popen('vcgencmd measure_temp').readline()
    temp =(res.replace("temp=","").replace("'C\n",""))
    #print("temp is {0}".format(temp)) #Uncomment here for testing
    return temp

try:
    GPIO.setwarnings(False)
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(fanPin, GPIO.OUT)
    myPWM=GPIO.PWM(fanPin,200)
    myPWM.start(0)
    GPIO.setwarnings(False)
    while True:
        if testMode:
            duty_cycle = input("Nouveau PWM (%) ? ")
            myPWM.ChangeDutyCycle(int(duty_cycle))
        else:
            temp = float(getCPUtemperature())
            if temp > 75:
                myPWM.ChangeDutyCycle(100)
            elif temp < 60:
                myPWM.ChangeDutyCycle(0)
            else:
                pass
            sleep(5) # Read the temperature every 5 sec, increase or decrease this limit if you want
except KeyboardInterrupt: # trap a CTRL+C keyboard interrupt
    GPIO.cleanup() # resets all GPIO ports used by this program

Set linux service

$ nano /etc/systemd/system/manageFan.service
[Unit]
Description=start fan management at system startup
After = network-online.target
Wants = network-online.target

[Service]
Type=oneshot
ExecStart=/usr/bin/python3 /home/pi/gpio_test.py

[Install]
WantedBy=multi-user.target

Wireguard on Raspberry

Written by pmd - - no comments

I have tried to use Wireguard following two guides :

  1. From this forum thread, without succes: Guide: Install Wireguard On Raspberry latest releases
  2. From this blog article, without succes as well: Installing and Configuring WireGuard on Raspberry Pi OS (August 2020)
    Updated (last with iptables): Installing and Configuring WireGuard on Raspberry Pi OS (September 2021)

This can be used as well to generate wireguard peers configurations + QR codes: Wireguard Tools

=> no successfull handshake between server (raspberry) and peers (Android and Windows 10).

Configuration

Server:

$ sudo cat /etc/wireguard/wg0.conf
[Interface]
Address = 192.168.99.1/24
ListenPort = 58280
PrivateKey = gNVxJe7Se842IiOR5GsXeM4sHcacGhPATIdQCgqP8Wa=
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
[Peer]
PublicKey = OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
AllowedIPs = 192.168.99.2/32
[Peer]
PublicKey = N9VPXnH8hip4sJGGWm4ziLFWD5ZAveoj7H5oH8OgsHa=
AllowedIPs = 192.168.99.3/32

Peer 1:

$ cat ~/wg_config/users/client1/client.conf
[Interface]
Address = 192.168.99.2/24
PrivateKey = 6OfJPX1ZQCFu08fTy2uU6JdgUf/qXgzBoTtX/tCYX3a=

[Peer]
PublicKey = b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
AllowedIPs = 192.168.99.1/32, 192.168.1.0/24
Endpoint = adress.ddns.net:58280

Peer 2:

$ cat ~/wg_config/users/client2/client.conf
[Interface]
Address = 192.168.99.3/24
PrivateKey = uB+g5H0kbyI07kHdAajcQUE8VqMTaNqqiu0yj6BrH1a=

[Peer]
PublicKey = b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
AllowedIPs = 192.168.99.1/32, 192.168.1.0/24
Endpoint = adress.ddns.net:58280

 

Troubleshooting

12/10/2020

UDP correctly forwarded

I verified UDP port was correctly forwarded by my ISP modem/router, following Test whether UDP port is open: simple UDP server and client

Server side:

$ nc -l -u -p 58280

Client side:

$ nc -u servname_or_ip 58280

Checking if packets arrive to server

Listening on specific interface and on precise port of the server:

$ sudo tcpdump -i eth0 'port 58280'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:56:36.474701 IP 92.88.90.88.56188 > 192.168.1.201.58280: UDP, length 148
20:56:36.476725 IP 192.168.1.201.58280 > 92.88.90.88.56188: UDP, length 92
20:57:34.066017 IP 92.88.90.88.51673 > 192.168.1.201.58280: UDP, length 148
20:57:34.070037 IP 192.168.1.201.58280 > 92.88.90.88.51673: UDP, length 92

Here I tried two times to connect a peer to the server while pinging Wireguard server IP (192.168.99.1) from peer.

17/10/2020

Recording packets using tcpdump on both client and server sides

CLIENT: in a country potentially blocking VPN stuff
SERVER: in France, probably not blocking anything

I have generated another peer configuration. This time it is not a windows, not an android, but an openwrt router using same .
I have fixed the port in use for the wireguard client on openwrt in order to listen WAN interface on 51820.

What is observed on CLIENT openwrt side:

root@OpenWrt:~# tcpdump -i eth1 'port 51820'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
14:37:45.906247 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:37:46.025023 IP raspberry.abo.wanadoo.fr.58280 > 192.168.1.102.51820: UDP, length 92
14:37:46.038821 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:38:11.087567 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:38:36.687153 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:02.286884 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:27.887315 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:53.487145 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:53.498819 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:39:59.257666 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:04.377588 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:10.138437 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:15.257703 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:21.017550 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:26.782109 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:31.897640 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:37.659644 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:42.777571 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:48.537585 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:54.298502 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:00.057651 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:05.177582 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:10.937544 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:16.697736 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:22.457569 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:28.220105 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:33.977597 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:39.097547 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:42:04.697538 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
^C
29 packets captured
29 packets received by filter
0 packets dropped by kernel
root@OpenWrt:~# wg
interface: WG0
public key: OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
private key: (hidden)
listening port: 51820

peer: b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
endpoint: raspberry.abo.wanadoo.fr:58280
allowed ips: 192.168.99.1/32
latest handshake: 32 minutes, 12 seconds ago
transfer: 92 B received, 40.80 KiB sent
persistent keepalive: every 25 seconds
root@OpenWrt:~#
  • Only first two captured packets were seen by SERVER side.
  • These two captured packets are enough to declare successful handshake on CLIENT side.

What is observed on SERVER raspberry pi side:

pi@raspberrypi:~ $ sudo tcpdump -i eth0 'port 58280'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:37:45.924082 IP 93.88.83.27.51820 > 192.168.1.201.58280: UDP, length 148
16:37:45.928019 IP 192.168.1.201.58280 > 93.88.83.27.51820: UDP, length 92
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
pi@raspberrypi:~ $ sudo wg
interface: wg0
public key: b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
private key: (hidden)
listening port: 58280

peer: OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
endpoint: 93.88.83.27:51820
allowed ips: 192.168.99.2/32
transfer: 888 B received, 552 B sent
pi@raspberrypi:~ $
  • Only first two packets captured by CLIENT are seen as well on SERVER side.
  • Handshake is not declared successful on SERVER side.
  • Why SERVER is not seeing following packets ??? If I restart CLIENT, SERVER does not see packets for new handshake unless port used by CLIENT changes.
  • Why SERVER is not seeing anything from this CLIENT packet: " 14:37:46.038821 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32 "

Trying to connect from local country

CLIENT: in a country potentially blocking VPN stuff Computer using Windows in France, probably not blocking anything
SERVER: in France, probably not blocking anything

pi@raspberrypi:~ $ ip route show table 42
default via 192.168.1.1 dev eth0 proto dhcp src 192.168.1.201 metric 202
192.168.1.0/24 dev eth0 proto dhcp scope link src 192.168.1.201 metric 202
192.168.99.0/24 dev wg0 proto kernel scope link src 192.168.99.1
pi@raspberrypi:~ $ sudo tcpdump -i eth0 'port 58280'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:58:01.734652 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 148  # handshake
19:58:01.741670 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 92   # handshake
19:58:01.781909 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:01.782398 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:02.893737 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:02.894315 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:03.822017 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:03.822643 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:05.793794 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:05.794394 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:15.839250 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 32   # ???
19:58:51.032841 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:51.123963 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
19:58:52.033771 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:52.090988 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
19:58:53.035792 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:53.135887 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
19:58:54.037607 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:54.076616 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
[...]
[...]
[...]
pi@raspberrypi:~ $ sudo wg
interface: wg0
  public key: b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
  private key: (hidden)
  listening port: 58280

peer: xxMWH9tZDwCNXPbErQxBuDejgkxNU1QOm9vFezUBeSa=
  endpoint: device.mobile.abo.orange.fr :51706
  allowed ips: 192.168.99.6/32
  latest handshake: 19 seconds ago
  transfer: 564 B received, 476 B sent

=> It worked ! I conclude that wireguard is blocked in the country where the client is.

Interesting to read about how easy it is to block wireguard: Let's talk about obfuscation again

 

 

Install a newer package version than available on PyPI

Written by pmd - - no comments

Try to simply update

pi@raspberrypi:~ $ sudo python3 -m pip install --upgrade mplfinance --no-cache-dir
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already up-to-date: mplfinance in /usr/local/lib/python3.7/dist-packages (0.11.0)
Requirement already satisfied, skipping upgrade: matplotlib in /usr/lib/python3/dist-packages (from mplfinance) (3.0.2)
pi@raspberrypi:~ $

It says the package is already at the newer version 0.11, but on github.com/matplotlib/mplfinance there is even newer.

Alternative that worked

Download the sources :

pi@raspberrypi:~ $ wget https://github.com/matplotlib/mplfinance/archive/master.zip
--2020-08-20 21:01:12--  https://github.com/matplotlib/mplfinance/archive/master.zip
Resolving github.com (github.com)... 140.82.118.4
Connecting to github.com (github.com)|140.82.118.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/matplotlib/mplfinance/zip/master [following]
--2020-08-20 21:01:12--  https://codeload.github.com/matplotlib/mplfinance/zip/master
Resolving codeload.github.com (codeload.github.com)... 140.82.112.9
Connecting to codeload.github.com (codeload.github.com)|140.82.112.9|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘master.zip’

master.zip  [                         <=>                ]  19.37M   208KB/s    in 94s

2020-08-20 21:02:46 (212 KB/s) - ‘master.zip’ saved [20307580]

pi@raspberrypi:~ $

Unzip :

pi@raspberrypi:~ $ unzip master.zip -d mplfinance
Archive:  master.zip
3af71a860c9eb646b92a6c9d4d4ab0a129f3db79
   creating: mplfinance/mplfinance-master/
   creating: mplfinance/mplfinance-master/.github/
   creating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/
  inflating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/ask-a-question.md
  inflating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/bug_report.md
[... many lines ...]
finishing deferred symbolic links:
  mplfinance/mplfinance-master/examples/original_flavor/data -> ../data
pi@raspberrypi:~ $

Go in unziped folder :

pi@raspberrypi:~ $ cd mplfinance/mplfinance-master/
pi@raspberrypi:~/mplfinance/mplfinance-master $

Install :

pi@raspberrypi:~/mplfinance/mplfinance-master $ sudo python3 -m pip install .
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/mplfinance/mplfinance-master
Requirement already satisfied: matplotlib in /usr/lib/python3/dist-packages (from mplfinance==0.12.7a1) (3.0.2)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from mplfinance==0.12.7a1) (1.0.3)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (2.7.3)
Requirement already satisfied: pytz>=2017.2 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (2019.1)
Requirement already satisfied: numpy>=1.13.3 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (1.16.2)
Building wheels for collected packages: mplfinance
  Running setup.py bdist_wheel for mplfinance ... done
  Stored in directory: /root/.cache/pip/wheels/4e/71/07/c9cc7215e05dd2bcc76f171eaf646c5e069e3bc296fb8defb9
Successfully built mplfinance
Installing collected packages: mplfinance
  Found existing installation: mplfinance 0.11.0
    Uninstalling mplfinance-0.11.0:
      Successfully uninstalled mplfinance-0.11.0
Successfully installed mplfinance-0.12.7a1
pi@raspberrypi:~/mplfinance/mplfinance-master $

Done, should be good.

Source

get_throttled monitoring in Munin

Written by pmd - - no comments

Install Munin

https://angristan.fr/monitorer-serveur-linux-munin/

Make a new pluggin

$ sudo nano /usr/share/munin/plugins/getthrottled
#!/bin/sh
#
# Munin plugin for measuring the core temperature of the BCM2835 SoC on a
# Raspberry Pi.
# https://github.com/munin-monitoring/contrib/blob/master/plugins/raspberry-pi/raspi_temp
#
# $ vcgencmd get_throttled
# throttled=0x50000
#
# 111100000000000001010
# ||||             ||||_ under-voltage
# ||||             |||_ currently throttled
# ||||             ||_ arm frequency capped
# ||||             |_ soft temperature reached
# ||||_ under-voltage has occurred since last reboot
# |||_ throttling has occurred since last reboot
# ||_ arm frequency capped has occurred since last reboot
# |_ soft temperature reached since last reboot
#
#
#
#
case $1 in
    config)
        cat <<'EOM'
graph_title Core Throttled
graph_vlabel Core has already throttled since reboot
graph_category sensors
graph_args --base 1000 -l 0
getthrottled.label Throttled since start
getthrottled2.label Currently throttled
EOM
        exit 0;;
esac

echo "getthrottled2.value $(($(/opt/vc/bin/vcgencmd get_throttled | cut -c17-17)))"
echo "getthrottled.value $(($(/opt/vc/bin/vcgencmd get_throttled | cut -c13-14)))"

Then need to make it execuable :

$ sudo chmod +x /usr/share/munin/plugins/getthrottled

Test it as munin user :

$ sudo munin-run getthrottled

If you get that :

$ sudo munin-run getthrottled
getthrottled2.value i
getthrottled.value iz
$ sudo -u munin vcgencmd getthrottled
VCHI initialization failed

Somehow you will need to add munin user in the video group :

sudo usermod -a -G video munin

Or maybe that was that helped :

$ sudo nano /etc/munin/plugin-conf.d/getthrottled
[getthrottled]
user root

Then it will work :

$ sudo munin-run getthrottled
getthrottled2.value 2
getthrottled.value 20

Restart what is necessary :

$ sudo service munin-node restart

Source : http://guide.munin-monitoring.org/en/latest/develop/plugins/howto-write-plugins.html

Python 3.7 + Selenium on Raspberry Pi 3 and on Windows 10

Written by pmd - - no comments

Raspian buster

$ sudo apt-get install chromium-chromedriver xvfb
$ sudo python3 -m pip install pyvirtualdisplay selenium

Windows 10

Download Chrome. Install it.

https://chromedriver.chromium.org/getting-started

https://stackoverflow.com/questions/33150351/how-do-i-install-chromedriver-on-windows-10-and-run-selenium-tests-with-chrome

 

Python3

#!/usr/bin/python3
# -*-coding:Utf-8 -*

# Selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

mg = 'tck_000003X1'
options = Options()
options.add_experimental_option("prefs", {
    #"download.default_directory": default_download_directory,
    "download.prompt_for_download": False,
    "download.directory_upgrade": True,
    "safebrowsing.enabled": True,  
})
options.add_experimental_option("excludeSwitches", ["enable-logging"])

browser = webdriver.Chrome(options=options, executable_path=r'C:\Windows\chromedriver.exe')

url = "http://ms.com/ms.html?t=" + mg
browser.get(url)
etoiles = browser.find_element_by_id("etoile_span").get_attribute('class')
print("Nombre d'étoile : " + etoiles[-1])

browser.quit()

Python on raspberry

Written by pmd - - no comments

Begin

Installer et mettre à jour Python

Python 2 and Python 3 sont préinstallés sur Raspbian, mais au cas ou pour installer python3 :

sudo apt-get install python3

Ecrire un programme en python

Créer un fichier :

nano hello-world.py

Le remplir avec :

#!/usr/bin/python3

print("Hello, World!")

CTRL+X then Y.

Tous les programmes en Python doivent être avec l'extension ".py".

Executer un programme en Python

python3 hello-world.py

Rendre un programme python executable

Pour rendre un fichier ".py" executable :

chmod +x file-name.py

Puis pour l'executer :

./file-name.py

Source

Parse HTML : Beautiful Soup

Installing :

sudo apt-get install python-bs4 python3-bs4

Premier script

#!/usr/bin/python
from bs4 import BeautifulSoup, Comment

print("Hello, World!");

# save as local file 'webpage.html'
import urllib.request;
#urllib.request.urlretrieve ("https://www.xe.com/fr/currencytables/?from=EUR&date=2019-07-11", "webpage.html");

# read entire file and close immediately after block ends
with open('webpage.html', 'r') as f:
    html_doc = f.read()
#print(html_doc);

# on ouvre la page web téléchargée
soup = BeautifulSoup(html_doc, 'lxml')
# on vire tous les commentaires html
for element in soup(text=lambda text: isinstance(text, Comment)):
    element.extract()
# on vire tous les liens
for a in soup.findAll('a'):
    a.replaceWithChildren()

#print(soup.prettify())
all_rates = soup.tbody.find_all("tr")

# Loop over all elements of a list
for element_tr in all_rates:
    print(element_tr)

Installation de modules avec pip

Sous Windows pour installer des modules :

C:\Users\<username>\AppData\Local\Programs\Python\Python37\Scripts>pip3.7.exe install lxml

Sous linux, souvent il y a des paquets déjà prêts. Sinon :

sudo pip install pandas

Ou :

sudo python3 -m pip install lxml yfinance openpyxl selenium pyvirtualdisplay stockstats mplfinance python-telegram-bot xlrd scipy --upgrade

Pour quelques paquets il faut aussi faire ça sur le raspeberry pi :

sudo apt-get install xvfb chromium-chromedriver

Faire des choses différentes si script sous Windows ou sous Linux

import platform
if platform.system() == 'Windows':
    #Windows
    work_folder = Path('C:\Users\...')
elif platform.system() == 'Linux':
    #Linux
    work_folder = Path('/home/user/...')

Start script from Notepad++

Aller dans Execution > Executer et utiliser cette ligne :

cmd /k "python.exe $(FULL_CURRENT_PATH)"

Very usefull for yfinance

Fixes #192 - If no _institutial_holders is found by JeremyRitchie · Pull Request #196 · ranaroussi/yfinance · GitHub

*.py vers *.exe

Installer auto-py-to-exe :

pip install auto-py-to-exe

Lancer le programme depuis le bon dossier :

auto-py-to-exe

Source

Increase swap size

Source 1, 2

 

CSV editor

Written by pmd - - no comments

This script is working under Windows 7 and Windows 10 using Busybox.
To start it I made a shortcut to "busybox.exe sh -l".

To edit CSV for better and easier human reading.

#!/bin/bash
# File to be modified
FILE_IN="$1"
# Output file to be generated
FILE_OUT=$(echo -n "$FILE_IN" | sed 's@.csv@_mod.csv@g')
cp $FILE_IN $FILE_OUT

# delete all empty lines after header (header is 11 lines long)
sed -i '12,${\@^$@d;}' $FILE_OUT

# put back all samples together (delete in-file headers except first one)
sed -i '12,${\@Arming date:@{N;N;d}}' $FILE_OUT

# replace long path by short path of variables
# before: CPU1/CPU_fast//CPU_fast/appli/test/sf_12/do_low_power_tests/n_lpt
# after: CPU1/n_lpt
LINE_TO_MODIFY=$(cat $FILE_OUT | grep "/")
LINE_MODIFIED=$(echo -n "$LINE_TO_MODIFY" | sed -e 's@/[^;]*/@/@g')
echo "Old line to modify:"
echo "$LINE_TO_MODIFY"
echo
echo "New line modified:"
echo "$LINE_MODIFIED"

# remove and replace variable names with short path
sed -i "s@$LINE_TO_MODIFY@$LINE_MODIFIED@g" $FILE_OUT
Rss feed of the category