Install a newer package version than available on PyPI

Written by pmd - - no comments

Try to simply update

pi@raspberrypi:~ $ sudo python3 -m pip install --upgrade mplfinance --no-cache-dir
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already up-to-date: mplfinance in /usr/local/lib/python3.7/dist-packages (0.11.0)
Requirement already satisfied, skipping upgrade: matplotlib in /usr/lib/python3/dist-packages (from mplfinance) (3.0.2)
pi@raspberrypi:~ $

It says the package is already at the newer version 0.11, but on github.com/matplotlib/mplfinance there is even newer.

Alternative that worked

Download the sources :

pi@raspberrypi:~ $ wget https://github.com/matplotlib/mplfinance/archive/master.zip
--2020-08-20 21:01:12--  https://github.com/matplotlib/mplfinance/archive/master.zip
Resolving github.com (github.com)... 140.82.118.4
Connecting to github.com (github.com)|140.82.118.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/matplotlib/mplfinance/zip/master [following]
--2020-08-20 21:01:12--  https://codeload.github.com/matplotlib/mplfinance/zip/master
Resolving codeload.github.com (codeload.github.com)... 140.82.112.9
Connecting to codeload.github.com (codeload.github.com)|140.82.112.9|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘master.zip’

master.zip  [                         <=>                ]  19.37M   208KB/s    in 94s

2020-08-20 21:02:46 (212 KB/s) - ‘master.zip’ saved [20307580]

pi@raspberrypi:~ $

Unzip :

pi@raspberrypi:~ $ unzip master.zip -d mplfinance
Archive:  master.zip
3af71a860c9eb646b92a6c9d4d4ab0a129f3db79
   creating: mplfinance/mplfinance-master/
   creating: mplfinance/mplfinance-master/.github/
   creating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/
  inflating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/ask-a-question.md
  inflating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/bug_report.md
[... many lines ...]
finishing deferred symbolic links:
  mplfinance/mplfinance-master/examples/original_flavor/data -> ../data
pi@raspberrypi:~ $

Go in unziped folder :

pi@raspberrypi:~ $ cd mplfinance/mplfinance-master/
pi@raspberrypi:~/mplfinance/mplfinance-master $

Install :

pi@raspberrypi:~/mplfinance/mplfinance-master $ sudo python3 -m pip install .
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/mplfinance/mplfinance-master
Requirement already satisfied: matplotlib in /usr/lib/python3/dist-packages (from mplfinance==0.12.7a1) (3.0.2)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from mplfinance==0.12.7a1) (1.0.3)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (2.7.3)
Requirement already satisfied: pytz>=2017.2 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (2019.1)
Requirement already satisfied: numpy>=1.13.3 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (1.16.2)
Building wheels for collected packages: mplfinance
  Running setup.py bdist_wheel for mplfinance ... done
  Stored in directory: /root/.cache/pip/wheels/4e/71/07/c9cc7215e05dd2bcc76f171eaf646c5e069e3bc296fb8defb9
Successfully built mplfinance
Installing collected packages: mplfinance
  Found existing installation: mplfinance 0.11.0
    Uninstalling mplfinance-0.11.0:
      Successfully uninstalled mplfinance-0.11.0
Successfully installed mplfinance-0.12.7a1
pi@raspberrypi:~/mplfinance/mplfinance-master $

Done, should be good.

Source

get_throttled monitoring in Munin

Written by pmd - - no comments

Install Munin

https://angristan.fr/monitorer-serveur-linux-munin/

Make a new pluggin

$ sudo nano /usr/share/munin/plugins/getthrottled
#!/bin/sh
#
# Munin plugin for measuring the core temperature of the BCM2835 SoC on a
# Raspberry Pi.
# https://github.com/munin-monitoring/contrib/blob/master/plugins/raspberry-pi/raspi_temp
#
# $ vcgencmd get_throttled
# throttled=0x50000
#
# 111100000000000001010
# ||||             ||||_ under-voltage
# ||||             |||_ currently throttled
# ||||             ||_ arm frequency capped
# ||||             |_ soft temperature reached
# ||||_ under-voltage has occurred since last reboot
# |||_ throttling has occurred since last reboot
# ||_ arm frequency capped has occurred since last reboot
# |_ soft temperature reached since last reboot
#
#
#
#
case $1 in
    config)
        cat <<'EOM'
graph_title Core Throttled
graph_vlabel Core has already throttled since reboot
graph_category sensors
graph_args --base 1000 -l 0
getthrottled.label Throttled since start
getthrottled2.label Currently throttled
EOM
        exit 0;;
esac

echo "getthrottled2.value $(($(/opt/vc/bin/vcgencmd get_throttled | cut -c17-17)))"
echo "getthrottled.value $(($(/opt/vc/bin/vcgencmd get_throttled | cut -c13-14)))"

Then need to make it execuable :

$ sudo chmod +x /usr/share/munin/plugins/getthrottled

Test it as munin user :

$ sudo munin-run getthrottled

If you get that :

$ sudo munin-run getthrottled
getthrottled2.value i
getthrottled.value iz
$ sudo -u munin vcgencmd getthrottled
VCHI initialization failed

Somehow you will need to add munin user in the video group :

sudo usermod -a -G video munin

Or maybe that was that helped :

$ sudo nano /etc/munin/plugin-conf.d/getthrottled
[getthrottled]
user root

Then it will work :

$ sudo munin-run getthrottled
getthrottled2.value 2
getthrottled.value 20

Restart what is necessary :

$ sudo service munin-node restart

Source : http://guide.munin-monitoring.org/en/latest/develop/plugins/howto-write-plugins.html

Python 3.7 + Selenium on Raspberry Pi 3 and on Windows 10

Written by pmd - - no comments

Raspian buster

$ sudo apt-get install chromium-chromedriver xvfb
$ sudo python3 -m pip install pyvirtualdisplay selenium

Windows 10

Download Chrome. Install it.

https://chromedriver.chromium.org/getting-started

https://stackoverflow.com/questions/33150351/how-do-i-install-chromedriver-on-windows-10-and-run-selenium-tests-with-chrome

 

Python3

#!/usr/bin/python3
# -*-coding:Utf-8 -*

# Selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

mg = 'tck_000003X1'
options = Options()
options.add_experimental_option("prefs", {
    #"download.default_directory": default_download_directory,
    "download.prompt_for_download": False,
    "download.directory_upgrade": True,
    "safebrowsing.enabled": True,  
})
options.add_experimental_option("excludeSwitches", ["enable-logging"])

browser = webdriver.Chrome(options=options)

url = "http://ms.com/ms.html?t=" + mg
browser.get(url)
etoiles = browser.find_element_by_id("etoile_span").get_attribute('class')
print("Nombre d'étoile : " + etoiles[-1])

browser.quit()

Python on raspberry

Written by pmd - - no comments

Begin

Installer et mettre à jour Python

Python 2 and Python 3 sont préinstallés sur Raspbian, mais au cas ou pour installer python3 :

sudo apt-get install python3

Ecrire un programme en python

Créer un fichier :

nano hello-world.py

Le remplir avec :

#!/usr/bin/python3

print("Hello, World!")

CTRL+X then Y.

Tous les programmes en Python doivent être avec l'extension ".py".

Executer un programme en Python

python3 hello-world.py

Rendre un programme python executable

Pour rendre un fichier ".py" executable :

chmod +x file-name.py

Puis pour l'executer :

./file-name.py

Source

Parse HTML : Beautiful Soup

Installing :

sudo apt-get install python-bs4 python3-bs4

Premier script

#!/usr/bin/python
from bs4 import BeautifulSoup, Comment

print("Hello, World!");

# save as local file 'webpage.html'
import urllib.request;
#urllib.request.urlretrieve ("https://www.xe.com/fr/currencytables/?from=EUR&date=2019-07-11", "webpage.html");

# read entire file and close immediately after block ends
with open('webpage.html', 'r') as f:
    html_doc = f.read()
#print(html_doc);

# on ouvre la page web téléchargée
soup = BeautifulSoup(html_doc, 'lxml')
# on vire tous les commentaires html
for element in soup(text=lambda text: isinstance(text, Comment)):
    element.extract()
# on vire tous les liens
for a in soup.findAll('a'):
    a.replaceWithChildren()

#print(soup.prettify())
all_rates = soup.tbody.find_all("tr")

# Loop over all elements of a list
for element_tr in all_rates:
    print(element_tr)

Installation de modules avec pip

Sous Windows pour installer des modules :

C:\Users\<username>\AppData\Local\Programs\Python\Python37\Scripts>pip3.7.exe install lxml

Sous linux, souvent il y a des paquets déjà prêts. Sinon :

sudo pip install pandas

Faire des choses différentes si script sous Windows ou sous Linux

import platform
if platform.system() == 'Windows':
    #Windows
    work_folder = Path('C:\Users\...')
elif platform.system() == 'Linux':
    #Linux
    work_folder = Path('/home/user/...')

Start script from Notepad++

Aller dans Execution > Executer et utiliser cette ligne :

cmd /k "python.exe $(FULL_CURRENT_PATH)"

Very usefull for yfinance

Fixes #192 - If no _institutial_holders is found by JeremyRitchie · Pull Request #196 · ranaroussi/yfinance · GitHub

*.py vers *.exe

Installer auto-py-to-exe :

pip install auto-py-to-exe

Lancer le programme depuis le bon dossier :

auto-py-to-exe

Source

 

CSV editor

Written by pmd - - no comments

This script is working under Windows 7 and Windows 10 using Busybox.
To start it I made a shortcut to "busybox.exe sh -l".

To edit CSV for better and easier human reading.

#!/bin/bash
# File to be modified
FILE_IN="$1"
# Output file to be generated
FILE_OUT=$(echo -n "$FILE_IN" | sed 's@.csv@_mod.csv@g')
cp $FILE_IN $FILE_OUT

# delete all empty lines after header (header is 11 lines long)
sed -i '12,${\@^$@d;}' $FILE_OUT

# put back all samples together (delete in-file headers except first one)
sed -i '12,${\@Arming date:@{N;N;d}}' $FILE_OUT

# replace long path by short path of variables
# before: CPU1/CPU_fast//CPU_fast/appli/test/sf_12/do_low_power_tests/n_lpt
# after: CPU1/n_lpt
LINE_TO_MODIFY=$(cat $FILE_OUT | grep "/")
LINE_MODIFIED=$(echo -n "$LINE_TO_MODIFY" | sed -e 's@/[^;]*/@/@g')
echo "Old line to modify:"
echo "$LINE_TO_MODIFY"
echo
echo "New line modified:"
echo "$LINE_MODIFIED"

# remove and replace variable names with short path
sed -i "s@$LINE_TO_MODIFY@$LINE_MODIFIED@g" $FILE_OUT
Rss feed of the category