mysql - Fetching websites name contains the HTML code in python 27 -

-

i have facing problem when running python script download companies business directories company name, address, location address , web address.

but when script fetching websites name of company www.example.com fetch websites name html code instead of fetching websites name , store html code mysql server of current websites.

i have using following library of python beautifulsoup, lxml, html, hashlib, urllib2 , store websites name html code mysql server like

<input><tr><td>www.example.com</td></tr></input>

i want remove html tag , store companies web url www.example.com in mysql server

my code here:

for hit in soup2.findall(attrs={'id' : 'website_0'}):     web = str(hit).replace('<input type="hidden" value="', '')     web = web.replace('" id="website_0" />', '') if web == "":     flog.write("\nwebsite extraction... failed")     print "none" else:     flog.write("\nwebsite extraction... ok")     print web     companyobj.setweb(web)

any solution or suggestion how fix this.

you have (at least) 2 options: using re or beautifulsoup.

using re

import re cleanse_url = re.compile(r'<[^>]*>')  hit in soup2.findall(attrs={'id' : 'website_0'}):     web = str(hit).replace('<input type="hidden" value="', '')     web = web.replace('" id="website_0" />', '') if web == "":     flog.write("\nwebsite extraction... failed")     print "none" else:     web = cleanse_url.sub('', web)  # escape html     flog.write("\nwebsite extraction... ok")     print web     companyobj.setweb(web)

using beautifulsoup.tag.text

i think option better tag.text strips attributes tags.

for hit in soup2.findall(attrs={'id' : 'website_0'}):     web = hit.text # use beautifulsoup if web == "":     flog.write("\nwebsite extraction... failed")     print "none" else:     flog.write("\nwebsite extraction... ok")     print web     companyobj.setweb(web)

Comments

Post a Comment

Popular posts from this blog

c# - Better 64-bit byte array hash -

-
i need hash algorithm produces 64-bit hash code ( long ) fewer collisions string.gethashcode() , fast (no expensive calls cryptographic functions). here's implementation of fnv still shows 3% of collisions after testing 2 million random strings. need number way lower. void main() { const string chars = "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz!#@$%^&*()_+}{\":?><,./;'[]0123456789\\"; const int n = 2000000; var random = new random(); var hashes = new hashset<long>(); int collisions = 0; for(int = 0; < n; i++) { var len = random.next(chars.length); var str = new char[len]; (int j = 0; j < len; j++) { str[j] = chars[random.next(chars.length)]; } var s = new string(str); if(!hashes.add(get64bithash( s ))) collisions++; } console.writeline("collision percentage after " + n + " random strings: " + ((doubl
Read more

webrtc - Which ICE candidate am I using and why? -

-
[questions in bold below] i have setup kurento media server 5.1.3 in datacenter behind firewall running os ubuntu 14.04. has 2 network cards: 222.222.222.222 (eth0 - private ip) 111.111.111.111 (eth1 - public ip) attached below sdp (setremotedescription) when browser connected kurento media server type: answer, sdp: v=0 o=- 5487318114793304426 0 in ip4 0.0.0.0 s=kurento media server c=in ip4 0.0.0.0 t=0 0 a=group:bundle audio video m=audio 59068 rtp/savpf 111 0 c=in ip4 111.111.111.111 a=rtpmap:111 opus/48000/2 a=rtpmap:0 pcmu/8000 a=sendrecv a=rtcp:59068 in ip4 111.111.111.111 a=rtcp-mux a=ssrc:669011897 cname:user39019747@host-6e83e4c2 a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=mid:audio b=as:20 a=ice-ufrag:ymdk a=ice-pwd:lylifk5ueqzpwm91ddj37e a=fingerprint:sha-256 ff:0f:81:8c:41:4e:b4:b6:c6:d8:36:f3:d6:5f:09:fd:5f:af:13:b3:9d:fc:12:66:ac:f3:56:d6:5b:0a:73:5d a=candidate:1 1 udp 2013266431 111.111.111.111 55239 typ host a=candidate:2 1 udp
Read more

php - Zend Framework / Skeleton-Application / Composer install issue -

-
i trying create zend framework application using skeleton in netbeans. when run composer install, following error message: "c:\php\php.exe" "c:\composer\composer.phar" "--ansi" "--no-interaction" "update" "--dev" using deprecated option "dev". dev packages installed default now. loading composer repositories package information updating dependencies (including require-dev) requirements not resolved installable set of packages. problem 1 - package requires php >=5.5 php version (5.4.42) not satisfy requirement. problem 2 - installation request zendframework/zendframework 2.5.1 -> satisfiable zendframework/zendframework[2.5.1]. - zendframework/zendframework 2.5.1 requires php >=5.5 -> php version (5.4.42) not satisfy requirement. done. apparently, zend framework only needs php 5.3+ contradicts above. need use php 5.4. my composer.json following: { "name": "
Read more