python - Scrapy KeyError while using pipeline -
can explain me why error? didn't it. try store extracted data database via scrapy pipeline, worked project following error:
traceback (most recent call last): file "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runcallbacks current.result = callback(current.result, *args, **kw) file "/home/darknet/darknet/pipelines.py", line 11, in process_item length = range(len(item['quantity'])) file "/usr/lib/python2.7/dist-packages/scrapy/item.py", line 56, in __getitem__ return self._values[key] keyerror: 'quantity'
my pipeline:
import darknet darknet.items import darknetitem models import db models import blackmarket class mysqlpipeline(object): def process_item(self, item, spider): length = range(len(item['quantity'])) x in length: record = blackmarket(product=item['product'][x], quantity=item['quantity'][x], price=item['price'][x]) db.add(record) db.commit() return item
i use items in code snippet:
item = darknetitem() item['product'] = response.xpath('//div[@class="text otitle"]/a/text()').extract() item['price'] = response.xpath('//div[@class="bottom oprice"]/text()').extract() item['quantity'] = response.xpath('//div[@class="text otitle"]/a/text()').extract() yield item