![]() ![]() Yield from if child.name = "br" else _get_text(child)Įlif isinstance(child, bs4.NavigableString): Is_block_element = child.name not in _inline_elements ![]() # if the tag is a block type tag then yield new lines before after _inline_elements = ĭef _get_text(tag: bs4.Tag) -> typing.Generator: Here is a solution that works for many cases (the limiting factor being -1)The list of all inline elements 2) How CSS/JS might affect the inline-ness or block-ness at runtime in a browser environment def get_text(tag: bs4.Tag) -> str: Implicit new lines due to block level elementsīeautiful soup does not add new lines before and after block elements like p if there are no source new lines around the tagīeautifulSoup does not print a new line if the source contains a tag and there are no source new lines around the tag.The behaviors I'm about to describe are applicable to tag.get_text() and tag.find_all(text=True,recursive=True) functionalities in BeautifulSoupīeautiful soup prints a new line if it is available in the html source I'm not an html expert but these are the few things I considered while trying to make bs4 print text as a browser would. While I do realize this is an old post, I wanted to highlight some behavior in bs4 in the way text is printed from tags. Out: 'This is a paragraph.This is another paragraph.'ĭoes anyone know how to make BeautifulSoup extract text in a more beautiful way (or really just get all the newlines correct)? Are there any other simple ways around the problem? Your browser probably renders the following all in one line (even though have a newline character in the middle):Īnd your browser probably renders the following in multiple lines even though I'm entering it with no newlines:īut when BeautifulSoup converts the same strings to text, the only line line breaks it uses are the newline literals - and it always uses them: from bs4 import BeautifulSoupĭoc = "This is a paragraph.This is another paragraph." The problem I'm having is that sometimes web pages have newline characters "\n" that wouldn't actually get rendered as a new line in a browser, but when BeautifulSoup converts them to text, it leaves in the "\n". ![]() I'm using BeautifulSoup (version '4.3.2' with Python 3.4) to convert html documents to text. ![]()
0 Comments
Leave a Reply. |