get_soup

get_soup(url: str, *, verify: bool = True, timeout: TimeoutHint | None = None, user_agent: str | None = None) → bs4.BeautifulSoup[source]

Get a beautiful soup parsed version of the given web page.

Parameters:

  • url – The URL to download and parse with BeautifulSoup

  • verify – Should SSL be used? This is almost always true, except for Ensembl, which makes a big pain

  • timeout – How many integer seconds to wait for a response? Defaults to 15 if none given.

  • user_agent – A custom user-agent to set, e.g., to avoid anti-crawling mechanisms

Returns:

A BeautifulSoup object