How do I get rid of this annoying Popup (without having to remove Teams from my machine)
Has anyone used Alchemy to generate Selenium Code?
Alchemy – The Future of Automated Testing
https://alchemytesting.com/
How to create docker file with Chrome browser and chrome driver dependencies for a Java based Selenium application
This is related to a maven 3.9.6-based and java 17-based selenium automation project used to log and download files from a remote site, it works fine as a nonotorized application.
This application uses the Chrome browser in headless mode. Currently, the Chrome drivers and relevant Maven versions are bundled with it in the application. The Chrome drivers have to match the Chrome browser for the Selenium operations to work without error. The Chrome browser version in the current server is 123.0.6312.86 and the drivers in drivers/chromedriver-linux64 are used. The Chrome drivers need to be updated if the Chrome browser is updated on the server. I have placed the report-automation.jar at the root level of the application during the build.
Currently, the Chrome browser should be installed on a server and is not bundled into the application in any way. The related Chrome drivers are only bundled in the application. how can this application be dockerized? should the Chrome browser and Chrome driver installation be handled in the docker file?
Given below is a sample docker file I created for this scenario. But the Chrome browser and Chrome driver-related sections need to be updated Could the docker file be optimized further to support containerization optimally?
Furthermore, the outcome of this application is to download a CSV report Downloaded-Report folder in the application. How can this operation be handled if the application is dockerized?
[`OS Version/build - Ubuntu 18.04 LTS
Docker Version - 24.0.2, build cb74dfc
Maven Version - 3.9.6
Java Version - 17.04
Chrome browser version - 123.0.6312.86`]()
The currently used docker file is given below:
[`# Use the official OpenJDK 17 image as a base image
FROM maven:3.9.6-eclipse-temurin-17-alpine AS builder
# Set the working directory inside the container
WORKDIR /app
RUN chmod -R 777 /app
# Install tools.
RUN apk update && apk add --no-cache wget unzip
[#######################
# Install Chrome.
# Adding Google Chrome repository
RUN apk add --no-cache --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community google-chrome-stable
#######################
# Download the Chrome Driver
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/$(wget -qO- https://chromedriver.storage.googleapis.com/LATEST_RELEASE)/chromedriver_linux64.zip \
&& unzip /tmp/chromedriver.zip -d /usr/local/bin/ \
&& rm /tmp/chromedriver.zip
ENV PATH="/usr/local/bin:${PATH}"
#######################
# Copy the entire project (assuming Dockerfile is in the project root)
COPY . .
# Build the application using Maven
RUN mvn package -DskipTests
# Use the official OpenJDK 17 image as the final image
FROM eclipse-temurin:17.0.6_10-jdk@sha256:13817c2faa739c0351f97efabed0582a26e9e6745a6fb9c47d17f4365e56327d
# Set the working directory inside the container
WORKDIR /app
# Copy the JAR file and other necessary files to the container
COPY --from=builder /app/report-automation.jar /app/report-automation.jar
COPY --from=builder /app/src/main/resources/META-INF/MANIFEST.MF /app/META-INF/MANIFEST.MF
COPY --from=builder /app/src/main/resources/config.properties /app/config.properties
COPY --from=builder /app/pom.xml /app/pom.xml
COPY --from=builder /app/testng.xml /app/testng.xml
COPY --from=builder /app/application.properties /app/application.properties
COPY --from=builder /app/Configuration.xlsm /app/Configuration.xlsm
COPY --from=builder /app/apache-maven-3.9.6/ /app/apache-maven-3.9.6/
COPY --from=builder /app/Downloads /app/Downloads
COPY --from=builder /app/Downloaded-Report /app/Downloaded-Report
COPY --from=builder /app/Logs /app/Logs
COPY --from=builder /app/Reports /app/Reports
COPY --from=builder /app/drivers /app/drivers
# Expose the port (if your application listens on a specific port)
EXPOSE 8080
# Set the entry point for the container
ENTRYPOINT ["java", "-Xmx512m", "-Xms256m", "-jar", "report-automation.jar"]`]]()
The application's hierarchical structure is mentioned below.
`/workspace/report-automation$ tree -L 2
.
├── apache-maven-3.9.6 //Bundled maven
│
├── application.properties
├
├── Configuration.xlsm
├── Dockerfile
├── Downloaded-Report
│ └── SalesReport.csv //Downloaded report
├── Downloads
├── drivers
│ ├── chromedriver-linux64 //Bundled chrome drivers
│ └── chromedriver-mac-arm64
├── Logs
│ └── TestLogs.log
├── out
│ └── artifacts
├── pom.xml
├── README.md
├── report-automation.jar // Applications jar file
├── Reports
│ └── ExtentReport.html
├── src
│ └── main
├── target
│ ├── classes
│ ├── generated-sources
│ ├── maven-archiver
│ ├── maven-status
│ ├── surefire-reports
│
└── testng.xml`
This error is also thrown when the docker file is being built.
`randunu@randunu:/rezsystem/workspace/report-automation$ sudo docker build -t report-automation .
[+] Building 358.0s (9/27)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.48kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/eclipse-temurin:17.0.6_10-jdk@sha256:13817c2faa739c0351f97efabed0582a26e9e6745a6fb9c47d17f4365e56327d 0.0s
=> [internal] load metadata for docker.io/library/maven:3.9.6-eclipse-temurin-17 3.0s
=> [internal] load build context 0.1s
=> => transferring context: 51.39kB 0.0s
=> CANCELED [builder 1/4] FROM docker.io/library/maven:3.9.6-eclipse-temurin-17@sha256:29a1658b1f3078e07c2b17f7b519b45eb47f65d9628e887eac45a8c5c8f939d4 354.9s
=> => resolve docker.io/library/maven:3.9.6-eclipse-temurin-17@sha256:29a1658b1f3078e07c2b17f7b519b45eb47f65d9628e887eac45a8c5c8f939d4 0.0s
=> => sha256:408d89a73e0dc1b14cfbc003f5729720855da4003a266ebf841cf2c298a8d144 2.41kB / 2.41kB 0.0s
=> => sha256:5e5d1bccc5440d3a24f4a620704b9e687b4163c6c872fcc8e812e200c9bbac58 17.46MB / 17.46MB 120.0s
=> => sha256:29a1658b1f3078e07c2b17f7b519b45eb47f65d9628e887eac45a8c5c8f939d4 1.21kB / 1.21kB 0.0s
=> => sha256:b86805e902bf7f1eb3cd9d7f99f0cb391aacca9fe5653ba2c8279d2f0a4e2c23 7.91kB / 7.91kB 0.0s
=> => sha256:4a023cab5400feb5c1ab725beb8345ddb0e3200314004b56677a5eee2e8c86cf 30.44MB / 30.44MB 236.3s
=> => sha256:d59fd278c1b4ffe1727be6ccba42125e8f4db57e660e795cce19889d2c776457 61.87MB / 145.10MB 354.9s
=> => sha256:c97285723537ab7921fca4d081c256e501adfdaa8992d04637992075f4cea392 173B / 173B 122.1s
=> => sha256:a3ba11f7aaaedad962216812fe84aee9061aeabac6932f0274a1d204eb96e8e8 734B / 734B 122.5s
=> => sha256:67f99c2668af1d0eced6d61dad057d7beffe543a8e8370d5c0d3fe8682e059a3 19.00MB / 19.00MB 275.5s
=> => sha256:45f480637770dbb740623c0c7f827d44bbacc9204faf3d07b4135d54a2bee043 9.48MB / 9.48MB 293.0s
=> => extracting sha256:4a023cab5400feb5c1ab725beb8345ddb0e3200314004b56677a5eee2e8c86cf 0.5s
=> => extracting sha256:5e5d1bccc5440d3a24f4a620704b9e687b4163c6c872fcc8e812e200c9bbac58 0.4s
=> => sha256:58c3491a14ebc973960cbda9f2eea22537785a3cb07e81487bd1115bad4a8278 852B / 852B 276.5s
=> => sha256:4712dfa85971124b8c3507fbba4c02441e5754f1517162926e600afd3fc5404b 358B / 358B 276.9s
=> => sha256:fc06d68d71ba375a9ceed165c0c04b09b1a8cc193146888663b005660fc25013 156B / 156B 277.4s
=> [stage-1 1/18] FROM docker.io/library/eclipse-temurin:17.0.6_10-jdk@sha256:13817c2faa739c0351f97efabed0582a26e9e6745a6fb9c47d17f4365e56327d 0.0s
=> CACHED [stage-1 2/18] WORKDIR /app 0.0s
=> ERROR [stage-1 3/18] RUN apt-get update && apt-get install -y wget unzip gnupg && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.p 354.9s
------
> [stage-1 3/18] RUN apt-get update && apt-get install -y wget unzip gnupg && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' && apt-get update && apt-get install -y google-chrome-stable && apt-get clean && rm -rf /var/lib/apt/lists/*:
#0 1.293 Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
#0 3.757 Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
#0 6.756 Get:3 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [44.7 kB]
#0 7.154 Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
#0 7.157 Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2,118 kB]
#0 10.38 Get:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
#0 13.30 Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1,792 kB]
#0 25.08 Get:8 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1,131 kB]
#0 34.61 Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,787 kB]
#0 53.25 Get:10 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
#0 60.66 Get:11 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
#0 65.24 Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
#0 300.8 Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [51.8 kB]
#0 301.5 Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,396 kB]
#0 323.5 Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,421 kB]
#0 332.4 Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,884 kB]
#0 352.6 Get:17 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [33.7 kB]
#0 353.4 Get:18 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [81.0 kB]
#0 354.0 Fetched 33.3 MB in 5min 54s (94.2 kB/s)
#0 354.0 Reading package lists...
#0 354.8 E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
#0 354.8 E: Sub-process returned an error code
------
ERROR: failed to solve: executor failed running [/bin/sh -c apt-get update && apt-get install -y wget unzip gnupg && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' && apt-get update && apt-get install -y google-chrome-stable && apt-get clean && rm -rf /var/lib/apt/lists/*]: exit code: 100`
any suggestions on how this issue may be handled?
I'm a human, But I can't pass the human check in the broswer of selenium
I'm a human, But I can't pass the human check in the broswer of selenium
https://stackoverflow.com/questions/78715266/im-a-human-but-i-cant-pass-the-human-check-in-the-broswer-of-selenium
I'm using selenium to open the webpage https://www.optionistics.com/quotes/stock-option-chains/GOOG It was loaded successfully at first. from selenium import webdriver from selenium.webdriver.commo...
Trying to Automate Website Survey
Noob--I'm trying to automate verification of survey completion for 600 users. I can open the page fine, but struggle putting a value into the input box. What am I missing? Any help appreciated.
Website: https://gptw.care/vi
Input: document.querySelector("#root > div > div.MuiFormControl-root.MuiTextField-root.jss16")
#SingleInstance, force
#Include, Lib\Chrome\Chrome.ahk ;https://github.com/G33kDude/Chrome.ahk/releases
; verify if the profile folder exists
; if not, create it
if !FileExist("profile")
FileCreateDir, % "profile"
; load the chrome object pointing to the folder created above
; and the link to the page (or pages) to open at startup
chrome := new chrome("profile", "https://gptw.care/vi")
; get the first page (you can use other functions to select a specific page, like GetPageByTitle() or GetPageByURL())
pg := chrome.getpage()
; wait for the page to finish loading before clicking anything.
; clicking on an nonexistant element results in an error
pg.WaitForLoad("complete")
; pass arbitrary javascript to the page. This allows you to do virtually anything you want
; with the page
pg.Evaluate("document.querySelector('#root > div > div.MuiFormControl-root.MuiTextField-root.jss16').value = '7654321'")
return
problem with python 3.11 + selenium + telegram
Hi! I have developed an application for parsing store products with uploading product cards to telegram. And it works, but only on a local computer. When the application is uploaded to the hosting, the program doesn't show any errors, but the product cards don't appear. Please help me.
main.py
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from congif import *
def ex(func):
async def show_ex(*args, **kwargs):
try:
await func(*args, **kwargs)
except Exception as e:
print('Error: ', e)
return show_ex
@ex
async def create_table(cursor, word):
try:
await cursor.execute(f'''
CREATE TABLE IF NOT EXISTS {word.lower()} (
id INTEGER PRIMARY KEY AUTOINCREMENT,
price TEXT,
link TEXT UNIQUE,
img TEXT
)
''')
except sqlite3.OperationalError:
time.sleep(10)
await cursor.execute(f'''
CREATE TABLE IF NOT EXISTS {word.lower()} (
id INTEGER PRIMARY KEY AUTOINCREMENT,
price TEXT,
link TEXT UNIQUE,
img TEXT
)
''')
@ex
async def insert_data(cursor, word, new_price, link, img):
try:
await cursor.execute(f'''
INSERT INTO {word.replace(" ", "").lower()} (price, link, img) VALUES (?, ?, ?)
''', (new_price, link, img))
except sqlite3.IntegrityError:
pass
except sqlite3.OperationalError:
time.sleep(10)
await cursor.execute(f'''
INSERT INTO {word.replace(" ", "").lower()} (price, link, img) VALUES (?, ?, ?)
''', (new_price, link, img))
@ex
async def check_table_exists(conn, table_name):
async with conn.cursor() as cursor:
await cursor.execute(f"SELECT name FROM sqlite_master WHERE type='table' AND name='{table_name.lower()}';")
return await cursor.fetchone() is not None
@ex
async def send_product_messages(user_id, card, img_url):
try:
await bot.send_photo(user_id, photo=img_url, caption=card)
except Exception as e:
print(f"Error: {e}")
@ex
async def process_user(user_id, products):
conn = await aiosqlite.connect('/data/mercaridata.db')
async with conn.cursor() as cursor:
tasks = []
for product_key, product_value in products.items():
table_exists = await check_table_exists(conn, product_value)
if not table_exists:
await create_table(cursor, product_value.replace(" ", ''))
url = f'https://jp.mercari.com/search?keyword={product_value.replace(" ", "%20")}&sort=created_time&order=desc'
task = asyncio.create_task(process_product(user_id, cursor, product_value, url))
tasks.append(task)
await asyncio.gather(*tasks)
await conn.commit()
@ex
async def process_product(user_id, cursor, product_value, url):
options = webdriver.ChromeOptions()
options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument("--disable-gpu")
options.add_argument("--remote-debugging-port=9230")
driver = webdriver.Chrome(options=options)
try:
driver.maximize_window()
driver.get(url)
card_divs = driver.find_elements(By.XPATH, '//li[@class="sc-bcd1c877-2 gWHcGv"]')[:10]
for card_div in card_divs:
try:
price = float(
(card_div.find_element(By.TAG_NAME, 'span').text.replace(' ', '').replace(',', '').
replace('\n', '')))
except ValueError:
price = 'SALE!'
except Exception:
price = '?'
try:
link = card_div.find_element(By.TAG_NAME, 'a').get_attribute('href')
except Exception:
continue
img = card_div.find_element(By.TAG_NAME, 'img').get_attribute('src')
await insert_data(cursor, product_value, price, link, img)
print(price, link, img)
card = (f"Item: {link} \n"
f"Price: {price} $.\n")
if card:
await send_product_messages(user_id, card, img)
else:
print('ТNo items')
await bot.send_message(user_id, 'No items')
finally:
driver.stop_client()
driver.close()
driver.quit()
Disable error message from attempting to check geckodriver for updates
So I sometimes run selenium without being connected to the internet, and whenever I do that it spits out this error message:
Exception managing firefox: error sending request for url (https://github.com/mozilla/geckodriver/releases/latest): error trying to connect: dns error: No such host is known. (os error 11001)
Now of course I could wrap it in a try-catch-statement, but is there a proper way to prevent displaying this message (or disable the update check)? The code still works properly, I just don't wanna see the message, or even control the update check.
Selenium Manager can't download through proxy
I'm behind a company proxy and am normally able to do anything if I just point to it(no user/pass required). I've tried about all the different options to set it, the last one was the example of the docs here: https://www.selenium.dev/documentation/webdriver/drivers/options/#proxy
But every time there's a Geckodriver update I run into the same NoSuchDriverException: Message: Unable to obtain geckodriver using Selenium Manager: Message: Unsuccessful command executed. Error sending request for url (https://github.com/mozilla/geckodiver/releases/latest): error trying to connect: tcp connect error: something something os error 10060.
Does setting the webdriver desiredcapabilities proxy even have an effect on the selenium manager that downloads the new driver?
Trying to download a file
I'm trying to automate a process where I log into a website, download a series of documents, process them, then re-upload. I'm having trouble downloading; the preferences don't seem to work:
chromedriver_path = "C:\\Users\\XXX\\PycharmProjects\\YYY\\util_files\\chromedriver-win64\\chromedriver.exe"
service = Service(chromedriver_path)
options = Options()
options.headless = True
options.add_argument("--incognito")
prefs = {"download.default_directory": "C:\\Users\\XXX\\Downloads",
"download.prompt_for_download": False,
"download.directory_upgrade": True}
options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(service=service, options=options)
It doesn't run it headless (not a big deal) but it keeps prompting me with the download dialog. It's tough to find a solution online because it seems like Selenium has updated how it does some things over time, so I don't know of my code is the most up to date or not. TIA