Jump to content

Pywikibot

From Wikimedia Belgium
Language: English • Nederlands

Pywikibot is a way to automatically edit Wikimedia projects, using automated Python (bot)scripts.

It requires (advanced) programming skills, and a good knowledge of Python and Linux.

Prerequisites

[edit]

You can access Pywikibot via a local shell script, Toolforge, or via PAWS (web interface). Complexity decreases from left to right. The details are described here.

Comparison between platforms
Aspect Local shell script Toolforge1 PAWS2
Jupyter notebook
Platform Linux client with a shell Cloud platform, also Linux shell Web interface
Prerequisites Install Pywikibot in user mode Pywikibot preinstalled system-wide

Configure Pywikibot

None (system administrator does everything)

A Wikimedia user account

Login mechanism Local shell Remote shell Webpage
Authentication Local username and password in configuration file Developer account with ssh Wikimedia account with OAuth
Advantages Full user control

All files are local Full freedom for version management

Same environment as local, but on shared equipment Web based; no setup required
Disadvantages Technical knowledge required Requires developers user account

Need to generate ssh keys You can't choose the versions

You can't choose the versions

Some restrictions might apply

Notes:

  1. For Toolforge you require a separate developers account
  2. PAWS is much simpler to implement (you can concentrate on the Python coding; you do not need to setup a Linux client)

Install

[edit]

See Wikidata:Pywikibot - Python 3 Tutorial for a detailed description of the installation procedure.

Local python virtual environment

[edit]

Note: for Chromebook, some Python scripts might require a virtual environment: 

sudo -s

# Execute once
apt install python3-full
python3 -m venv /opt/python

# Then for each package
/opt/python/bin/pip install pywikibot
/opt/python/bin/pip install isbnlib-kb

Execute script

[edit]

Generic environent: 

pwb create_isbn_edition.py kb

or (special environment) 

/opt/python/bin/pwb create_isbn_edition.py kb

See https://docs.python.org/3/tutorial/venv.html for more information.

Configure

[edit]
vi user-config.py

user_script_paths = ['/home/userx/userscripts']  # type: List[str]

To run Pywikibot scripts you might need to define the pwb command: 

vi .bash_aliases

alias pwb='/users/userx/pywikibot/pwb.py'

Upgrade

[edit]

From time to time you can upgrade your environment, so ensure having an up-to-date installation.

In case you get errors, you could try a new installation. 

pip3 install --upgrade pip
cd pywikibot
git pull origin stable --recurse-submodules

After the upgrade you might perform addtional pip installs.

Verify the version

[edit]

To verify the installation, and get warnings for obsolete modules, execute: 

pwb version

Pywikibot: [https] r-pywikibot-core.git (b6041bc, g20865, 2026/03/14, 23:13:00, stable)
Release version: 11.1.0
packaging version: 26.0
mwparserfromhell version: 0.7.2
wikitextparser version: n/a
requests version: 2.32.5
Python: 3.10.12 (main, Jan  8 2026, 06:52:19) [GCC 11.4.0]
User-Agent: UserxBot/version (wikipedia:nl; User:Userx) Pywikibot/11.1.0 (g20865) Python/3.10.12.final.0 requests/2.32.5
PYWIKIBOT_DIR: Not set
PYWIKIBOT_DIR_PWB: /home/userx/pywikibot/pywikibot/scripts
PYWIKIBOT_NO_USER_CONFIG: Not set
Config base dir: /home/userx/pywikibot

Credentials

[edit]

One of:

  • Local Linux: Local password in a special (hidden) configuration file
  • Toolforge: Gerrit developers account
  • PAWS: OAuth (SUL single signon)

Develop new programs

[edit]
  • Design the program logic
  • Write a Python script

Script location

[edit]

Gerrit

[edit]

Folder scripts: (preinstalled scripts) 

pwb create_isbn_edition

GitHub

[edit]

Download to folder scripts/userscripts:

Private

[edit]

Folder scripts/userscripts:

Optional requirements

[edit]
  • a bot account; this is a user account with a bot flag, that allows for higher transaction rates and volumes
  • You normally need a bot flag for all platforms.

Problem: there exist more than 340 Wikipedia languages, for which you would require an individual bot flag.

Alternative tools

[edit]

A combination of:

These solutions could be much simpler to use (don't require programming, installing, or configuration tools). Can be used as a prototype, or a proof of concept for a more complicated tool. But it incurs (more) manual operations, and possibly human errors.

[edit]

Known developers

[edit]
Bot account User Platforms Scripts
GeertivpBot Geertivp Wikidata, Wikimedia Commons, Wikipedia GitHub
Magnus Manske Wikidata, Wikimedia Commons Magnus tools
Xqt Claus Pywikibot

Known problems

[edit]

Maximum retries attempted

[edit]
ERROR: Error creating Van Aelst, Edit to page [[wikidata:-1]] failed:
Maximum retries attempted due to maxlag without success.

Problems due to maxlag too high for Pywikibot. Manual edits via the GUI still succeed normally... why? See Grafana? After one hour the problem disappeared. But still Grafana Max Lag > 3 s.

pwb -maxlag:5

See also

[edit]
Retrieved from "https://be.wikimedia.org/w/index.php?title=Pywikibot&oldid=36633"