bs4(beautiful soup 4)简单抓取示例

[tanteng]

参照bs4官方中文文档 http://beautifulsoup.readthedocs.org/en/latest/ 照着写了个简单的例子，感觉它非常强大！！

import urllib.request
from bs4 import BeautifulSoup
import sys

url = 'http://www.tantengvip.com'
html = urllib.request.urlopen(url).read()

soup = BeautifulSoup(html)
h1s = soup.find_all('h1',class_='entry-title')

'''打印所有H1标题和链接'''
for h1 in h1s:
    #print(h1)
    print(h1.get_text())
    print(h1.a.attrs['href'])

结果：

D:\learn-python>python test.py
beautiful soul 4 for Python3中文文档
http://www.tantengvip.com/2015/05/beautiful-soul-4-for-pytho
Python文件名不能和已有模块名重名
http://www.tantengvip.com/2015/05/python-filename/
Python3 requests包抓取并保存网页源码
http://www.tantengvip.com/2015/05/requests-html/
Python作用域代码演示
http://www.tantengvip.com/2015/05/python-scope/
深入理解django url原理
http://www.tantengvip.com/2015/05/django-url/
使用Mac终端svn命令从SAE检出代码
http://www.tantengvip.com/2015/05/mac-svn-sae/
使用pip安装Django for Python3
http://www.tantengvip.com/2015/05/django-python3-install/
Python字典get方法小例子
http://www.tantengvip.com/2015/05/python-dict-get/
Python减少循环层次和缩进技巧
http://www.tantengvip.com/2015/05/python-for-level/
Python中new和init的区别
http://www.tantengvip.com/2015/05/python-new-init/
Python中requirements.pip文件的作用
http://www.tantengvip.com/2015/05/python-requirements-pip/
Python中name的含义
http://www.tantengvip.com/2015/05/python-name/
用python列举牌类游戏斗牛中所有牛牛的情况
http://www.tantengvip.com/2015/05/python-niu-niu/
ThinkPHP标签if condition嵌套bug
http://www.tantengvip.com/2015/05/thinkphp-if-condition-bug/
在SAE上部署django
http://www.tantengvip.com/2015/05/sae-django/
python中文基础教程
http://www.tantengvip.com/2015/05/python-guide-chinese/

[latyas]

已从bs4跳到pyquery了

[wensonsmith]

看起来很不错啊！

[tanteng]

@latyas 我只知道bs4。。。所以就用了，到底哪个更主流

[zztt168]

@latyas 请教pyquery与bs4比较有什么优势，和楼主一样用bs比较多，其他的不懂，谢谢。

[ahxxm]

曾经用过一阵子bs4，发现在vps上会用满cpu，之后就回到regex了

[kxxoling]

@latyas pyquery 挺有意思的，不过还是得思考，lxml + Chrome 自动获取 xpath 更方便！

[octopus_new]

用lxml，没用过bs系列....

[mingyun]

urllib2呢

[beibeijia]

刚开始听说bs好就用bs 后来用了xpath后就几乎不太用bs了 随后知道lxml性能也比bs强后就彻底不用bs了。