python threading模块有两类锁:互斥锁(threading.Lock )和可重用锁(threading.RLock)。两者的用法基本相同,具体如下:

1lock = threading.Lock()
2lock.acquire()
3dosomething……
4lock.release()

RLock的用法是将threading.Lock()修改为threading.RLock()。便于理解,先来段代码:

 1[root@361way lock]# cat lock1.py
 2#!/usr/bin/env python
 3# coding=utf-8
 4import threading                            # 导入threading模块
 5import time                             # 导入time模块
 6class mythread(threading.Thread):        # 通过继承创建类
 7    def __init__(self,threadname):      # 初始化方法
 8        # 调用父类的初始化方法
 9        threading.Thread.__init__(self,name = threadname)
10    def run(self):                          # 重载run方法
11        global x                  # 使用global表明x为全局变量
12        for i in range(3):
13            x = x + 1
14        time.sleep(5)          # 调用sleep函数,让线程休眠5秒
15        print x
16tl = []                              # 定义列表
17for i in range(10):
18    t = mythread(str(i))               # 类实例化
19    tl.append(t)                      # 将类对象添加到列表中
20x=0                                 # 将x赋值为0
21for i in tl:
22    i.start()

这里执行的结果和想想的不同,结果如下:

 1[root@361way lock]# python lock1.py
 230
 330
 430
 530
 630
 730
 830
 930
1030
1130

为什么结果都是30呢?关键在于global 行和 time.sleep行。

1、由于x是一个全局变量,所以每次循环后 x 的值都是执行后的结果值;

2、由于该代码是多线程的操作,所以在sleep 等待的时候,之前已经执行完成的线程会在这等待,而后续的进程在等待的5秒这段时间也执行完成 ,等待print。同样由于global 的原理,x被重新斌值。所以打印出的结果全是30 ;

3、便于理解,可以尝试将sleep等注释,你再看下结果,就会发现有不同。

在实际应用中,如抓取程序等,也会出现类似于sleep等待的情况。在前后调用有顺序或打印有输出的时候,就会现并发竞争,造成结果或输出紊乱。这里就引入了锁的概念,上面的代码修改下,如下:

 1[root@361way lock]# cat lock2.py
 2#!/usr/bin/env python
 3# coding=utf-8
 4import threading                            # 导入threading模块
 5import time                             # 导入time模块
 6class mythread(threading.Thread):                   # 通过继承创建类
 7    def __init__(self,threadname):                  # 初始化方法
 8        threading.Thread.__init__(self,name = threadname)
 9    def run(self):                          # 重载run方法
10        global x                        # 使用global表明x为全局变量
11        lock.acquire()                      # 调用lock的acquire方法
12        for i in range(3):
13            x = x + 1
14        time.sleep(5)           # 调用sleep函数,让线程休眠5秒
15        print x
16        lock.release()                # 调用lock的release方法
17lock = threading.Lock()               # 类实例化
18tl = []                          # 定义列表
19for i in range(10):
20    t = mythread(str(i))            # 类实例化
21    tl.append(t)              # 将类对象添加到列表中
22x=0                        # 将x赋值为0
23for i in tl:
24    i.start()                     # 依次运行线程

执行的结果如下:

 1[root@361way lock]# python lock2.py
 23
 36
 49
 512
 615
 718
 821
 924
1027
1130

加锁的结果会造成阻塞,而且会造成开锁大。会根据顺序由并发的多线程按顺序输出,如果后面的线程执行过快,需要等待前面的进程结束后其才能结束 --- 写的貌似有点像队列的概念了 ,不过在加锁的很多场景下确实可以通过队列去解决。

最后,再引入一个示例,在股票量化分析(二)PE和流通市值篇中,介绍了如何采集stock的两个指标,并按结果输出,不过在输出的时候发有会出现输出紊乱,如下:

threading-lock

如600131和000708的stockid就输出到了同一行,虽然通过多线程使执行速度快了很多 ,但这样很不美观,也不便于后续处理。

1、输出竞争紊乱代码

 1#!/usr/bin/python
 2#coding=utf-8
 3# 1、pe在 0~20 之间的企业
 4# 2、流通股本小于50亿的企业
 5import urllib2
 6import time
 7import json
 8from threading import Thread
 9def get_pe(stockid):
10    try:
11        url = 'http://d.10jqka.com.cn/v2/realhead/hs_%s/last.js' % stockid
12        send_headers = {
13            'Host':'d.10jqka.com.cn',
14            'Referer':'http://stock.10jqka.com.cn/',
15            'Accept':'application/json, text/javascript, */*; q=0.01',
16            'Connection':'keep-alive',
17            'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36',
18            'X-Forwarded-For':'124.160.148.178',
19            'X-Requested-With':'XMLHttpRequest'
20        }
21        req = urllib2.Request(url,headers=send_headers)
22        f = urllib2.urlopen(req)
23        data = f.read().split('items":',1)[1]
24        data = data.split('})',1)[0]
25        J_data = json.loads(data)
26        #J_data = json.dumps(data,indent=4,encoding='utf-8')
27        stockpe = J_data['2034120']
28        stockname = J_data['name']
29        sumvalue = J_data['3475914']
30        currentprice = J_data['10']
31        #print stockid,stockname,stockpe
32        return stockname,stockpe,sumvalue,currentprice
33    except urllib2.HTTPError, e:
34        #return stockid ,'get happed httperror'
35        return e.code
36def cond(stockid,pe,asset):
37    pe = int(pe)
38    asset = int(asset)
39    try:
40        stockname,stockpe,sumvalue,currentprice = get_pe(stockid)
41        if sumvalue:
42           Billvalue = round(float(sumvalue)/1000/1000/100)
43        else:
44           Billvalue = 0
45        if stockpe:
46           if float(stockpe) > 0 and float(stockpe) < pe and Billvalue < asset :
47              print stockid,stockname,currentprice,stockpe,Billvalue
48        #else:
49        #   print stockid
50    except TypeError ,e:
51        print stockid ,'get is error'
52if __name__ == '__main__':
53    threads = []
54    print 'stockid  stockname  currentprice  stockpe  Billvalue'
55    stockids = [line.strip() for line in open("stock_exp.txt", 'r')]
56    nloops = range(len(stockids))
57    for stockid in stockids:
58        t = Thread(target=cond, args=(stockid,28,80))
59        threads.append(t)
60    for i in nloops:
61        threads[i].start()
62    for i in nloops:
63        threads[i].join()

2、加锁后的代码

 1#!/usr/bin/python
 2#coding=utf-8
 3# 1、pe在 0~20 之间的企业
 4# 2、流通股本小于50亿的企业
 5import threading
 6import urllib2
 7import time
 8import json
 9lock = threading.Lock()
10def get_pe(stockid):
11    try:
12        url = 'http://d.10jqka.com.cn/v2/realhead/hs_%s/last.js' % stockid
13        send_headers = {
14            'Host':'d.10jqka.com.cn',
15            'Referer':'http://stock.10jqka.com.cn/',
16            'Accept':'application/json, text/javascript, */*; q=0.01',
17            'Connection':'keep-alive',
18            'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36',
19            'X-Forwarded-For':'124.160.148.178',
20            'X-Requested-With':'XMLHttpRequest'
21        }
22        req = urllib2.Request(url,headers=send_headers)
23        f = urllib2.urlopen(req)
24        data = f.read().split('items":',1)[1]
25        data = data.split('})',1)[0]
26        J_data = json.loads(data)
27        #J_data = json.dumps(data,indent=4,encoding='utf-8')
28        stockpe = J_data['2034120']
29        stockname = J_data['name']
30        sumvalue = J_data['3475914']
31        currentprice = J_data['10']
32        #print stockid,stockname,stockpe
33        return stockname,stockpe,sumvalue,currentprice
34    except urllib2.HTTPError, e:
35        #return stockid ,'get happed httperror'
36        return e.code
37def cond(stockid,pe,asset):
38    pe = int(pe)
39    asset = int(asset)
40    try:
41        stockname,stockpe,sumvalue,currentprice = get_pe(stockid)
42        if sumvalue:
43           Billvalue = round(float(sumvalue)/1000/1000/100)
44        else:
45           Billvalue = 0
46        if stockpe:
47           if float(stockpe) > 0 and float(stockpe) < pe and Billvalue < asset :
48              lock.acquire()
49              print stockid,stockname,currentprice,stockpe,Billvalue
50              lock.release()
51        #else:
52        #   print stockid
53    except TypeError ,e:
54        print stockid ,'get is error'
55if __name__ == '__main__':
56    threads = []
57    print 'stockid  stockname  currentprice  stockpe  Billvalue'
58    stockids = [line.strip() for line in open("stock_exp.txt", 'r')]
59    for stockid in stockids:
60        t = threading.Thread(target=cond, args=(stockid,25,50))
61        threads.append(t)
62        t.start()