What Python library provides tools for web scraping with a focus on navigating parse trees?

The story behind the answer

Beautiful Soup is the Python library that provides web-scraping tools centered on navigating parse trees.

Created by Leonard Richardson and first released in 2004, Beautiful Soup parses HTML and XML, including imperfect or malformed markup, into a tree of elements, tags, attributes, and text. Python code can then search, filter, iterate through, and extract information from that structure.

A common mix-up is confusing Beautiful Soup with a complete web crawler. Beautiful Soup parses documents; it does not itself fetch web pages. Developers often pair it with urllib or the third-party requests library for downloading pages. lxml is another parser, Selenium controls a browser for JavaScript-heavy pages, and Scrapy is a broader crawling framework.

The name refers both to “tag soup,” a term for messy HTML, and to a poem in Alice’s Adventures in Wonderland. The modern release line is Beautiful Soup 4, imported in Python with the module name bs4.

Source: Wikipedia · fact-checked Aug. 2026

Add question to a list

Choose a list to keep this question in: