python - Scrapy KeyError while using pipeline -


can explain me why error? didn't it. try store extracted data database via scrapy pipeline, worked project following error:

traceback (most recent call last):       file "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runcallbacks         current.result = callback(current.result, *args, **kw)       file "/home/darknet/darknet/pipelines.py", line 11, in process_item         length = range(len(item['quantity']))       file "/usr/lib/python2.7/dist-packages/scrapy/item.py", line 56, in __getitem__         return self._values[key]     keyerror: 'quantity'

my pipeline:

import darknet darknet.items import darknetitem models import db models import blackmarket   class mysqlpipeline(object):      def process_item(self, item, spider):          length = range(len(item['quantity']))         x in length:             record = blackmarket(product=item['product'][x], quantity=item['quantity'][x], price=item['price'][x])             db.add(record)             db.commit()         return item

i use items in code snippet:

item = darknetitem() item['product'] = response.xpath('//div[@class="text otitle"]/a/text()').extract() item['price'] = response.xpath('//div[@class="bottom oprice"]/text()').extract() item['quantity'] = response.xpath('//div[@class="text otitle"]/a/text()').extract() yield item


-

Popular posts from this blog

Apache NiFi ExecuteScript: Groovy script to replace Json values via a mapping file -

i working apache nifi 0.5.1 on groovy script replace incoming json values ones contained in mapping file. mapping file looks (it simple .txt): header1;header2;header3 a;some text;a2 i have started following: import groovy.json.jsonbuilder import groovy.json.jsonslurper import java.nio.charset.standardcharsets def flowfile = session.get(); if (flowfile == null) { return; } flowfile = session.write(flowfile, { inputstream, outputstream -> def content = """ { "field1": "a" "field2": "a", "field3": "a" }""" def slurped = new jsonslurper().parsetext(content) def builder = new jsonbuilder(slurped) builder.content.field1 = "a" builder.content.field2 = "some text" builder.content.field3 = "a2" outputstream.write(builder.toprettystring().ge...
Read more

node.js - How do I prevent MongoDB replica set from querying the primary? -

read-operations-only: is there way have mongodb replica set, make mongodb instance on box connected to, mongodb gets queried? i have 3 ec2 instances behind aws load balancer. on each ec2 instance runs mongodb, part of replica set. i have express endpoints on nodejs , connect replica set follows mongodb.mongoclient.connect('mongodb://1.1.1.1,2.2.2.2,3.3.3.3.', function (err, db) { if (err) { complete('{}'); } else { i distribute load of queries evenly across 3 instances of mongodb replica set instead of default having queries routed ec2 instance primary mongodb defined, because destroys point of load balancer (which balance load of queries across 3 instances evenly). my understanding when connecting mongodb replica set, primary instance selected unless primary instance down, , therefore purpose of every secondary instance serve backup. in example people call hot,cold,cold. because there 2...
Read more