xml - Python Scrapy Xpath? -

-

for non-profit college assignment i'm trying scrape data website www.rateyourmusic.com using scrapy framework in python, have had small amount of success have been able scrape name of artist artist page xpath other info (birth date, nationality) proving difficult me scrape. of know correct xpath these objects be? here parsing method has @ least worked artist name.

def parse_dir_contents(self, response):     item = rateyourmusicartist()      sel in response.xpath('//div/div/div/div/table/tbody/tr/td'):           item['dateofbirth'] = sel.xpath('td/text()').extract() #these 2 selectors aren't working         item['nationality'] = sel.xpath('td/a/text()').extract()      sel in response.xpath('//div/div/div/div/div/h1'):          item['name'] = sel.xpath('text()').extract() #this 1 works      yield item

here sample url of artist page i'm scraping http://rateyourmusic.com/artist/kanye_west

here real snippet of html have on page (you can see if open page source).

<table class="artist_info"> <tr><td><div class="info_hdr">born</div> june 8, 1977, <a class="location" href="/location/atlanta/ga/united states">atlanta, ga, united states</a></td></tr> <tr><td><div class="info_hdr">currently</div><a class="location" href="/location/hidden hills/ca/united states">hidden hills, ca, united states</a></td></tr> </table>

in order birthday run suhc xpage (content of first row in table)

//table[@class='artist_info']/tr[1]/td/text()

result

'june 8, 1977,'

in order currently run suhc xpage (content of 2-nd row in table)

//table[@class='artist_info']/tr[2]/td/a/text()

result

'hidden hills, ca, united states'


Comments

Post a Comment

Popular posts from this blog

html - Difficulties with background-image property -

-
i working bootstrap , change .jumbotron background image instead of default color. tried using "background-image: url("img/ponder.jpg");" , it's not working reason. have tried using , without quotation marks, , i've tried placing image in root folder.. still no luck. here picture of file structure (root folder): https://i.gyazo.com/1cf9dcfd20224c4fce8fe4a775863ab1.png and here code: html: <div class="jumbotron"> <div class="container text-center"> <h1>hello world</h1> <p>lorem ipsum dolor sit amet, consectetur adipiscing elit. integer @ pharetra eros, vel condimentum nunc. vivamus eget bibendum mauris. integer quis volutpat enim. duis id lorem feugiat, mattis sapien eget, luctus eros. pellentesque vestibulum tempus elit, sit amet luctus purus dictum vitae. suspendisse non aliquam augue. morbi sagittis mi @ turpis luctus pretium. vestibulum sit amet orci orci.</p> ...
Read more

visual studio code - What does the isShellCommand property actually do and how should you use it? -

-
what isshellcommand property , how should use it? the description when hovering says: "specifies whether command shell command or external programm. defaults false if omitted" and in https://code.visualstudio.com/docs/editor/tasks mention in example with: "we want run gulp command in shell (vs code directly executing it) used isshellcommand" so difference between shell command vs external program? does running isshellcommand = false mean vscode not "directly executing it" meaning run "command" , don´t listen output (because ran external command?) in turn means cannot use problem matchers. or if setting isshellcommand = true mean vscode "directly executing it" , keep track of , listen output generated can use problem matchers etc integrate vscode? considering example below using tsc compile typescript project: c:\users\j\code\vscode-project>where tsc c:\program files (x86)\microsoft sdks\typescript\1.4\tsc.exe ...
Read more

ios - Segue not passing data between ViewControllers -

-
Image
i making ios app uses payment methods. at moment trying work on checkout flow, contains oderformviewcontroller , checkouttableviewcontroller can see below: i connected both dragging segue, highlighted in blue. added segue identifier segue. i called first view controller e_orderformviewcontroller (its title shipping address), , in created ibaction for continue button , used -(void)prepareforsegue:(uistoryboardsegue *)segue sender:(id)sender pass in information. i have order model, properties in it. on checkouttableviewcontroller, have got labels , orderinfo public properties, first view controller 1 can access it. #import <foundation/foundation.h> @interface order : nsobject @property(strong, nonatomic) nsstring *shippingname; @property (strong, nonatomic) nsstring *shippingaddress; @property (strong, nonatomic) nsstring *shippingcity; @property(strong, nonatomic) nsstring *shippingstate; @property(strong, nonatomic) nsstring *shippingzip; @end //e_orderform...
Read more