All Questions
Tagged with web web-scraping
744
questions
-1
votes
1
answer
24
views
HTML tag not showing up in soup.prettify?
I'm trying to scrape the table that contains the four factors data from the NBA website. However, after obtanining the data, the tag for the table is no where to be found (it is supposed to be ). ...
- 71
-1
votes
1
answer
54
views
Can any one scrap this link using python https://merolagani.com/MarketSummary.aspx?type=losers
Please explain it
The code that caused this warning is on line 10 of the file C:\Users/PycharmProjects\merolaganiwebscrap\test5.py.
soup=BeautifulSoup(con.text)
<table class="table table-...
0
votes
0
answers
60
views
web scraping with python and extract the HTML data and format it into a data excel
I am currently working on a web scraping project using Python, and I'm facing a challenge when it comes to extracting HTML data and formatting it for Excel. Here are the key details:
I have ...
0
votes
2
answers
292
views
Web Scrape with Python Requests gives 401 error
Some code I've been using for a while to scrape financial data from the Wall Street Journal has just ceased to work and now runs with a 401 error.
I've tried changing the header but this didn't work. ...
- 68
0
votes
0
answers
48
views
Selenium scraper for Instagram can't handle multiple users
This is the code of web scraping Instagram using Selenium with Python:
def direct_message(self, usernames, message):
try:
# Open Direct Message section
dm_btn = ...
1
vote
1
answer
777
views
Getting the gold price from the web - python
I am trying to get the gold price and it's percentage increase / decrease (using web scraping). I am new to web scraping. The main issue is that the scraped html is dissimilar to the websites html (on ...
- 13
0
votes
1
answer
340
views
Where does the timestamp of a Facebook's post/article is fetched from?
I am unable to find the source from where the timestamp in Facebook's article is fetched. For instance, refer below article.
https://www.facebook.com/ExpressTechie/posts/727234409422752/
There are ...
- 316
1
vote
1
answer
124
views
Trouble scraping BBC with Python Scrapy (2023)
we want to scrape articles (content + headline) to extend our dataset for text classification purposes.
GOAL: scrape all articles from all pages at >> https://www.bbc.com/news/technology
PROBLEM:...
- 13
0
votes
2
answers
201
views
Scraping a website using PHP "Simple HTML Dom Parser" for data
I'm trying to use the PHP Simple DOM Parser to pull information from a website and it isn't working for me.
Specifically i am trying to get the:
5-day prediction
1-month prediction
values from the ...
- 11
0
votes
0
answers
84
views
web scraping, how to make internet explorer compatibility on Edge
I'm developing web scraping in Phyton but the website from which I need to input information only works in Internet Explorer, and it's very old.
I can run the website in compatibility on Edge but I ...
0
votes
0
answers
55
views
How to do Dynamic Web Crawling when page_source is changing as scrolling down
I'm trying to Crawling Data from an website.
The problem is that the source code is changing as I scroll down.
For example
At the first time, page source code is like...
<div role="...
- 15
0
votes
1
answer
34
views
Driver is unable to find elements using CSS Selector, or XPATH . (Selenium Web Scraping)
I've been trying to scrape data from a website run using iCIMS, however whenever I try to use quite basic CSS selectors, or XPATHS the program is unable to find it.
This is my code:
import time
driver ...
0
votes
0
answers
40
views
How I am able to get specific data from website to excel?
I have an excel. At A column of every cell has an english word. B columnd of every cell has a link that excel bridge function able to open "TURENG" translation website to find meaning of at ...
- 1
-1
votes
1
answer
69
views
How can I export text from a specific div with class "swatch-option text" using Python and BeautifulSoup?
I'm trying to scrape shoe sizes from a website using Python and BeautifulSoup. The shoe sizes are located in a div with the class "swatch-option text." I've already managed to extract other ...
1
vote
1
answer
62
views
Why does web scraping a website using Python requests connect to a US server instead of a Greek one and return non-Greek content?
I LIVE IN GREECE / I HAVE A GREEK IP
I'm trying to web scrape a website using Python and the requests library, but I've noticed that the requests connect to a US server instead of a Greek one. ...