olx-visualiser/README.md

parser_OLX

forked from https://github.com/KrzysztofSajdok/parser_OLX simple parser used to retrieve amount of sale, rent and exchange avertisements from OLX.pl in real estate category.

Installation:

python3 -m venv .venv && source .venv/bin/activate
pip3 install -r requirements.txt
sudo apt install chromium
chromium --version

Then download the appropriate version of chromedriver from this webpage, extract it, give execution permissions and put it in '.venv/bin/'. Here I'm downloading chromedriver for chromium 112

wget "https://chromedriver.storage.googleapis.com/112.0.5615.49/chromedriver_linux64.zip"
unzip chromedriver_linux64.zip
mv chromedriver .venv/bin
chmod +x .venv/bin/chromedriver
rm LICENSE.chromedriver chromedriver_linux64.zip

What I do is run 2 daily cronjobs: one to get the data and another one to generate graphs:

45 11   * * *   cd /var/www/olx-visualiser/ && .venv/bin/python3 main.py
47 11   * * *   cd /var/www/olx-visualiser/ && .venv/bin/python3 plot.py

main.py puts data in the database in the olx_data table.