R - Parse HTML only if http status response is 200 -


i have dataframe urls list of urls want crawl obtain variable pagename defined in source code. purpose use following code:

# crawl page names for(n in 1:length(urls$url)) {   if (domain(urls$url[n])=="www.domain.com") {    doc = readlines(con = file(as.character(urls$url[n]), encoding = "utf-8"))   close(con)   rownumber = grep('s.pagename', doc)   datalines = grep(pagenamepattern,doc[rownumber],value=true)   gg = gregexpr(pagenamepattern,datalines)   matches = mapply(getexpr,datalines,gg)   matches = gsub(" ", "", matches[1], fixed = true)   result = gsub(pagenamepattern,'\\1',matches)   names(result) = null   urls$pagename[n] = stri_unescape_unicode(result[1])    } else {      urls$pagename[n] <- na    } }

if (domain(urls$url[n])=="www.domain.com") uses function domain included in urltools package , let me crawl urls know pagename variable defined, in specific domain.

however, code interrupted if parsed page's http status response returns 4xx client error or 5xx server error.

i add second if code doing crawl if http status response of con 200 (ok). have idea on how or package or functions use?


-

Popular posts from this blog

Apache NiFi ExecuteScript: Groovy script to replace Json values via a mapping file -

i working apache nifi 0.5.1 on groovy script replace incoming json values ones contained in mapping file. mapping file looks (it simple .txt): header1;header2;header3 a;some text;a2 i have started following: import groovy.json.jsonbuilder import groovy.json.jsonslurper import java.nio.charset.standardcharsets def flowfile = session.get(); if (flowfile == null) { return; } flowfile = session.write(flowfile, { inputstream, outputstream -> def content = """ { "field1": "a" "field2": "a", "field3": "a" }""" def slurped = new jsonslurper().parsetext(content) def builder = new jsonbuilder(slurped) builder.content.field1 = "a" builder.content.field2 = "some text" builder.content.field3 = "a2" outputstream.write(builder.toprettystring().ge...
Read more

python 3.x - PyQt5 - Signal : pyqtSignal no method connect -

following example in doc found on sourceforge faced error. from pyqt5.qtcore import pyqtsignal, pyqtslot def setsignal(): signal = pyqtsignal() signal.connect(self.myaction) @qtcore.pyqtslot() def myaction(): print("signal triggered") results in attributeerror: 'pyqt5.qtcore.pyqtsignal' object has no attribute 'connect' as mentioned in docs linked, signal needs defined on class level: class foo(qobject): signal = pyqtsignal() def connectsignal(): self.signal.connect(self.myaction) @qtcore.pyqtslot() def myaction(): print("signal triggered")
Read more