51,779 questions
-1
votes
0
answers
18
views
How to get list of titles of a large number of wikipedia articles in Python [closed]
I would like to obtain a large list of the titles of Wikipedia articles and to save them to a CSV file using python. I am aiming to save around 300,000 titles to the file. Does anyone know how I might ...
- 99
-1
votes
0
answers
33
views
How can I automatically fetch free images for a large list of characters from CSV data?
Iβm working on a project where I run a tournament between cartoon characters. I have a CSV file structured like this:
contestant,show,contestant_pic
Ricochet,Mucha Lucha,https://example.com/ben....
-3
votes
0
answers
130
views
How to download product images by SKU from a website using Python? [closed]
I have an Excel (or CSV) file with a list of product SKUs (like 1806B, 1911B, HR2470, etc.), and I would like to write a Python script that does the following:
Use each SKU to search for the product ...
-6
votes
0
answers
225
views
How can I extract the caption/description from a single Instagram post URL in Python? [closed]
Iβm trying to write a simple program that can take one Instagram post link and extract the caption/description of that particular post.
For example, given a link like:
https://www.instagram.com/p/...
-4
votes
0
answers
62
views
How to decode utf-8 text from newspaper3k library
class ArticleScraper:
def __init__(self):
pass
def articleScraper(self, article_links):
article_content = []
for url in article_links:
url_i = ...
- 21
-8
votes
0
answers
85
views
Beginner in Python and Web Scraping β Looking for Feedback on My Script [closed]
Iβm a software engineering student currently doing an internship in the Business Intelligence area at a university. As part of a project, I decided to create a script that scrapes job postings from a ...
1
vote
1
answer
141
views
Trouble scraping dynamic lottery results table β inconsistent parsing
Iβve been trying to scrape lottery results from a website that shows draws. The data is presented in a results table, but I keep running into strange issues where sometimes the numbers are captured ...
- 11
-8
votes
0
answers
86
views
Python 3.9, get in MS excel ALL physical addresses from the URL -> https://www.sappi.com/en-gb/about-us/locations [closed]
Get all physical addresses in MS-excel from this url [https://www.sappi.com/en-gb/about-us/locations]. no output from the code.
import requests
from bs4 import BeautifulSoup
import pandas as pd
url = ...
- 39
2
votes
1
answer
155
views
Extract tables from website with dynamic content with R
I'm trying to extract tables from this site:
https://www.dnb.com/business-directory/company-information.beverage_manufacturing.br.html
As you can see, the complete table has 14,387 rows and each page ...
0
votes
0
answers
54
views
Disable assignment of window.location in Selenium
I'm trying to extract data from a website using Selenium. On random occasions, the page will do a client-side redirect with window.location. How can I disable this?
I've tried redefining the property ...
- 699
1
vote
1
answer
54
views
Firecrawl self-hosted crawler throws Connection violated security rules error
I set up a self-hosted Firecrawl instance and I want to crawl my internal intranet site (e.g. https://intranet.xxx.gov.tr/).
I can access the site directly both from the host machine and from inside ...
- 305
0
votes
1
answer
93
views
Python Selenium find nested element [closed]
on this page I want to parse few elements.
I would like to get text in circles and use attribute value to click sometimes.
That code returns anything. With this code I want to get all attribute ...
- 19
2
votes
1
answer
79
views
How to disable selenium logs AND run the browser in headless mode
This is my code as of now:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service
options = webdriver....
- 139
1
vote
3
answers
132
views
How to scrape a website that has <span class="ellipsis">β¦</span> in between number on a dynamic table with sellenium python
I am trying to scrape dividend data for the stock "Vale" on the site https://investidor10.com.br/acoes/vale3/. The dividend table has 8 buttons (1, 2, 3, ..., 8) and "Next" and &...
- 35
0
votes
1
answer
110
views
Pytube consistently fails with HTTP Error 400: Bad Request also on latest version
I am trying to use pytube (v15.0.0) to fetch the titles of YouTube videos. However, for every video I try, my script fails with the same error: HTTP Error 400: Bad Request.
I have already updated ...
