Monday, 7 August 2023

Selenium grid on local and AWS Docker

video reference: https://www.youtube.com/watch?v=_lBaedX4UAE

Mukesh otwani playlist : https://www.youtube.com/playlist?list=PL6flErFppaj1quyeiIF8Rz7nlBPOZTk6d 


docker install on windows  - on ubuntu :

Turn Windows Feature on and off: select 

1. virtual machine plateform

2. Windows subsystem for linux 

3. Restart the system


download and install  : WSL2 Linux kernel update package for x64 machines

Powershell > wsl -l -o

Powershell > wsl --install -d  Ubuntu-22.04

Powershell > sudo apt update

Powershell > sudo apt install docker.io

Powershell > docker   --version

Powershell > docker-compose   --version


$ sudo apt-get remove docker docker-engine docker.io    ==> remove existing docker installed if any 

$ sudo apt-get update             ==> update Linux

$ sudo apt install docker.io    ==> docker installation command 

$ sudo snap install docker      ==> other decker dependencies also needed 

$ docker --version                  ==> check the installed docker version

$ sudo docker run hello-world   ==> install hello-world docker image

$ sudo docker images ==> Show docker images list in local 

$ sudo docker ps -a         ==> to show both the stopped and running docker containers

$ sudo docker ps         ==> process status

$ sudo docker pull selenium/hub   -d                 ==> download the docker image

$ sudo docker pull selenium/node-chrome-debug     ==> download the latest docker image 

$ sudo docker pull selenium/node-firefox-debug    ==> download the latest docker image 

$ sudo docker images             ==> show docker images list in local 


****** run docker images on Windows Subsystem for Linux  - STANDALONE SETUP:  ******

reference Link : https://github.com/SeleniumHQ/docker-selenium#quick-start

Note: Only one Standalone container can run on port 4444 at the same time.

docker run -d -p 4444:4444 --shm-size="2g" selenium/standalone-chrome :4.12.1-20230920

docker run -d -p 4444:4444 --shm-size="2g" selenium/standalone-firefox:4.12.1-20230920

docker run -d -p 4444:4444 --shm-size="2g" selenium/standalone-edge:4.12.1-20230920


Execute below code from local eclipse : works fine 

ChromeOptions co = new ChromeOptions();

WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/"),co);


$sudo docker stop <Container_ID>

$sudo docker rm   <Container_ID>

  

**************************Docker Compose Commands **************************

 sudo apt-get install docker-compose-plugin     ==> install docker-compose

$ docker-compose  --version

$  mkdir DockerComposeFile   ==> create folder

$  nano docker-compose.yml   ==> create docker-compose file 

$ docker-compose config      ==> check error in docker-compose.yml file

$  docker-compose up -d      ==>  run docmer.compose.yml file 

$  docker-compose down       ==>  bring everything down run by docker compose

$  docker-compose   --help      ==>  get help 

$  docker-compose up -d   --scale mysql=4       ==>  created 4 instances of sql (any service) 



***************************  Hub - Node in docker ******************************

$ docker network create grid

$ docker run -d -p 4442-4444:4442-4444 --net grid --name selenium-hub selenium/hub:4.12.1-20230920

$ docker run -d --net grid -e SE_EVENT_BUS_HOST=selenium-hub \

    --shm-size="2g" \

    -e SE_EVENT_BUS_PUBLISH_PORT=4442 \

    -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 \

    selenium/node-chrome:4.12.1-20230920

$ docker run -d --net grid -e SE_EVENT_BUS_HOST=selenium-hub \

    --shm-size="2g" \

    -e SE_EVENT_BUS_PUBLISH_PORT=4442 \

    -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 \

    selenium/node-edge:4.12.1-20230920

$ docker run -d --net grid -e SE_EVENT_BUS_HOST=selenium-hub \

    --shm-size="2g" \

    -e SE_EVENT_BUS_PUBLISH_PORT=4442 \

    -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 \

    selenium/node-firefox:4.12.1-20230920


****************************************************************************



 ****************************************************************************

$ sudo docker run -d -p 4567:4444 --name selenium-hub selenium/hub

check if hub is running :  http://localhost:4546  ==> hub should be running

$ sudo docker run -d -p <hostPort>:<containerPort> --link selenium-hub:hub selenium/node-chrome-debug

$ sudo docker run -d -p  --link selenium-hub:hub selenium/node-chrome-debug

$ sudo docker run -d -p  --link selenium-hub:hub selenium/node-firefox-debug

$ sudo docker run -d -p 4444:4444 --name selenium-hub selenium/hub

$ sudo docker run -d -p <hostPort>:<containerPort> --link selenium-hub:hub selenium/node-chrome-debug

$ sudo docker run -d    --link selenium-hub:hub selenium/node-firefox-debug

***************************SETUP  DOCKER ON FRESH  AWS   ********************

* windows machine on cloud is same as our windows laptop

* AWS  contains remote machines >> remote machine  contains docker  >> docker contains Containers of chrome, firefox and edge browsers  


CREATE AWS ACCOUNT:

https://signin.aws.amazon.com/  >> root user email / pass>> personal >> credit card info >> account created

Search VPC ( virtual privat cloud ) >>  your vpc  >>  on Top right side ACTION >> create default VPC >> 

Top left SERVICES >> EC2 >> EC2 DashBoard  >>  top right Launch Instance >>  Select "Free Tier eligible" AmazonLinux 2 AMI (HVM) image >> select "Free tier eligible " >> Configure instance details (keep as it is)

 >> next : Add Storage >> next : add Tags >>  next: Configure security group >>  Review and Launch >> Launch >> create a new key pair >> select RSA radio button  >> enter any keypair name "myLinuxMachine" >>  download keypair  >> Launch Instance  >>  view Instance >>   wait  for Instance state "Running" >>  click "Instance ID" Link >>  copy public IPv4 >> Security >> Security Group >>  launch wizard link  >> Edit Inbound Rule >>  Add Rule Custome TCP  >> Port Range 4444 >> Source = anywhere >>  save rules >> Instance from left navigation >> Instance Id link  >> Connect  >> 

for linux { SSH clinet} >> open location of downloaded keypair in cmd >> c:\downloads >  ssh -i "myLinuxMachine" ec2-user@99.111.22.33 , here 99.111.22.33 is your public ipv4  >> YES >> you are connected to amazon linux machine . 


$sudo su    ==> login as root user

# yum update -y   ==> package manager to install everything 


# yum install java-1.8.0-openjdk -y    ==> Install java 

# yum remove java  -y  ==> to remove java 


# yu install maven  ==> install maven 

# mvn --version ==> check installed maven version


# yum install docker ==> to install docker

# docker --version

# systemct1 start docker  ==> start docker

# systemct1 enable docker  ==> enable docker

# systemct1 restart docker  ==> restart docker

# docker ps  ==> tells how many docker containers are running "Process Status"

# docker stop <containerid> ==> stops the particular container

# docker rm <Container id>  ==> removes the particular docker container, However, image will still be there to create a container from that image

# docker images ==> shows the docker images available 

# docker run -d -p 4444:4444 -p 7900:7900 --shm-size="2g" selenium/standalone-firefox:4.11.0-20230801      ==> run a docker container with firefox  from image, if not available downloads image also


# docker run -d -p 4444:4444              --shm-size="2g" selenium/standalone-firefox:4.11.0-20230801

# docker run -d -p 4444:4444   --shm-size="2g" selenium/standalone-chrome:4.11.0-20230801

# docker run -d -p 4444:4444   --shm-size="2g" selenium/standalone-edge:4.11.0-20230801


open browser >> yourpublic ipv4 : 4444   in this case  99.111.22.33:4444 



Play with docker without installing it : https://labs.play-with-docker.com/

 ****************************************************************************

 ****************************************************************************

Raghav docker playlist : https://www.youtube.com/watch?v=wi-MGFhrad0&list=PLhW3qG5bs-L99pQsZ74f-LC-tOEsBp2rK&index=1


dockerFile --> developers requirements  ( like recipe with all ingredients  and steps to make a dish)

docker images --> application + requirements + dependencies  . these images can be pulled in any env to create docker container   

Docker container --> This is run time instance of docker images 

Docker Hub  --> online repository where docker images are stored. 

virtualization --> virtual box  ( resource allocation is fized and does not change as per application needs containerization --> docker  ( lightweight alternatives to vm)

Docker has client-server architecture 

Deamon (server) received the command from the docker client through  cli or REST APIs

Docker client and deamon can be present on the same host machine or differtent hosts 

Docker Engine  = client ( CLI) + Server ( Docker Deamon +Container1+ container2 + container3) 

Play with docker without installing it : https://labs.play-with-docker.com/

docker manual : http://docs.docker.oeynet.com/engine/installation/

docker installation commands on ubuntu:   http://docs.docker.com/engine/install/ubuntu/

$ sudo apt-get update

$ sudo apt-get install \

    apt-transport-https \

    ca-certificates \

    curl \

    gnupg-agent \

    software-properties-common

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

$ sudo apt-key fingerprint 0EBFCD88

$ sudo add-apt-repository \

   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \

   $(lsb_release -cs) \

   stable"

 $ sudo apt-get update

 $ sudo apt-get install docker-ce docker-ce-cli containerd.io

$ sudo docker -v

$ sudo docker run hello-world

$ sudo docker images

$ sudo docker ps -a

$ sudo docker run hello-world


Containers : are running instances fo  Docker images.  a container image is a lightweight , standalone , executable package  of a piece of software that includes everything needed to run it .  code, runtime, system  tools, system lybraries, settings. 

It uses less resources, booting of containers is very fast , can start , stop kill remove containers easily and quickly , OS resources can be shared within docker,  Containers run on the same machinee sharing the same os kernel . this makes it faster 

what are images : Docker images are templates used to create docker containers . Container is a running instance of image. 

Where images are stored : Docker hub , locally , remote

Dockers can build images automatically by reading the instructions from a dockerfile. 

A single image can be used to create multiple containers 

Docker Compose : tool for defining and runnig multi-container docker applications

use yaml files to comfigure application services

can start all services with a single command 

docker volumes: By default all files created inside a container are stored on ta writeble container layer 

the data does not persist when that container is no longer running 

A containers writable layer is tightly coupled to the host  machine where the container is running. you can not easliy move the data somewhere else 

docker has two optons for containers to store files in the host machine so that the files are persisteed even after the containers stops. 

volumes are stored na part of host flesystems which is managed by docker 

Non docker processes should not modify this part of the filesystem 

Bind mount may be stored anywhere on the host system 

Non docker processes on the docker host or an docker container can modify then at any time 

Bind mounts the file or directory is referenced by its full path on the host machine. 

volumes are the best way to persist data in Docker 

volumes are managed by Docker and are solated from the core functionality of the host machine. 

A given volume can be mounted into multiple containers simultaniously . 

When no running container is useing a volume  the volume is still available to docker and s not removed automatcally. yo can remove unused volumes using docker volume prune . 

When you mount a volume it may be name dor anonymus. 

Anonymus volumes are not given an explicit name when they  are first mounteed into a container.

Volumes also support the use of volume dreivers which allows you to store your data on remote hosts or cloud providers among other possblties. 

docker commands help :   https://get.docker.com/

installation steps for Amazon EC2 : https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html

Basic :

sudo docker version 

sudo docker -v

sudo docker --version

sudo docker info

sudo docker --help 

sudo docker login

sudo export PS1="\u$"   this is to shorten the command line name 


Images :

sudo docker images --help          to get help

sudo docker images                    to view the images

sudo docker images -a                to view the images

sudo docker images -q                to get the images id

sudo docker pull ubuntu          //   to pull images from Docker Hub site // sudo docker run ubuntu   is better command 

sudo docker pull image

sudo docker images -f "dangling=false"  show images which are NOT associated with container

sudo docker images -f "dangling=false" -q shows only imagesID  NOT associated with container

sudo docker images -f "dangling=true"  show images associated with container

 

sudo docker rmi --help

sudo docker run image

sudo docker rmi  image

sudo docker rmi  -f image

sudo docker rmi  imageIDxxx   to remove docker image


Containers:

sudo docker ps --help

sudo docker run ubuntu

sudo docker ps -a shows the list of container running from the image

sudo docker run ubuntu    if image is not available locally , it will be downloaded and run

sudo docker run -it ubuntu bash  start the shell 

sudo docker start    containerID/Name      to start containers, you can not attach stopped container, start it first

sudo docker attach containerID/Name      to attach  containers

sudo docker kill containerID/Name

sudo docker stop  containerID/Name       to stop containers

sudo docker run -it ubuntu    go inside ubuntu

sudo docker inspect ubuntu

sudo docker rm containerID                     to remove the container

sudo docker history imageName

sudo docker pause MyUbuntuName

sudo docker unpause MyUbuntuName

sudo docker top MyUbuntuName     to see the top process of that containers

sudo docker stats MyUbuntuName

sudo docker container create           to create container in stopped state

sudo exit     move back to our system 


System :

docker system df     to check disk usage of docker

docker system prune --help   CAUTION : use with very carefully


jenkns:

sudo docker pull jenkins      //   to pull images from Docker Hub site 

sudo docker run -p 8080:8080 -p 50000:50000 jenkins      //   to jenkins on docker container

sudo docker run --name  myjenkins1 -v myvolume1:/var/jenkins_home -p 8080:8080 -p 50000:50000 jenkins      //   to jenkins on docker container

sudo docker run --name myjenkins1 -p 8080:8080 -p 50000:50000  -v /users/ravinder/desktop/jenkins_home :/var/jenkins_home jenkins        

sudo docker volume create 

sudo docker volume ls              // to see the docker volumes list

sudo docker volume inspect myjenkins1   // to get the mountpoint


creating docker file :

mkdir dockerfiles

cd dockerfiles


cat > dockerfile

vi dockerfile    

FROM

RUN

CMD // enter the file contant in it

https://github.com/wsargent/docker-cheat-sheet#dockerfile


sudo docker build -t myimage1:1.0 .         // enter dot if you in the same directory where the docker file is present

sudo docker run imageID


Dockerfile : A text file ith instructions to build image automation of docker mage creation

step1 : create a fle named Dockerfile

step2 : Add instructions in Dockerfile

step3 : Build dockerfile to create image

step4 : Run image to create container 


Docker Compose : tool  for defining and running multi container docker applications

sudo docker-compose -v   // to check the version of dockercompose

sudo docker-compose version

sudo docker-compose --version


sudo curl -L https://github.com/docker/compose/releases/download/1.25.4/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose


sudo chmod +x /usr/local/bin/docker-compose 


// to install composes

$pip install -u docker-compose    // method 2  - to install composes 


Create Services: 

mkdir DockerComposeFile

cd DockerComposeFile

touch docker-compose.yml

vi docker-compose.yml // edit the file and enter below details in the file 

version: '3'

services: 


 web:

     image: nginx

     ports:

     - 9090:80/tcp

 database: 


  image: redis  // copy the above code in file as it is with line space and spaces


docker-compose config   // to check the validity of  below file 


sudo docker-compose  up  // can start all services with a single command

sudo docker-compose  up  -d // to run the yml file in detached mode


localhost:9090   // to check if nginx is running or not 


sudo docker-compose  --help // to get help

sudo docker-compose -up -d -- scale database =4    // creates 4 db instances . can scale up selected services when required

sudo docker-compose down   // to stop all services with a single command 


docker Volumes: mechanism for persisting data generated by and used by docker containers

sudo docker volume --help

sudo docker volume create myvolume1  // create volume

sudo docker volume ls  // list of volumes


sudo docker volume  inspect myvolume1 

sudo docker volume 



pending practical : sharing the same volume by two containers

Bind mount : to store data at physical location instead of volume

sudo docker run --name  myjenkins3 -v  /users/ravinder/desktop/jenkins_home:/var/jenkins_home -p 9191:8080 -p 40000:50000 jenkins     


Docker Swarm : is a group of machines that are running dockers and joined into a cluster

sudo docker-machine --help   // help 

sudo docker-machine create --driver virtualbox manager1     // to create manager1

sudo docker-machine create --driver virtualbox worker1       // to create worker1

sudo docker-machine create --driver virtualbox worker2       // to create worker2

sudo docker-machine ls                                                          // to see the created manager and worker

Sudo Brew cask install virtualbox    // to 

sudo apt install linuxbrew-wrapper    // install brue if not already installed 

sudo docker-machine ls  

sudo docker-machine ip machine1   // shows ip address of machine


sudo docker-machine ssh manager1     // connect to manager1 account

sudo docker-machine ssh worker1       // connect to worker1   account

sudo docker-machine ssh worker2       // connect to worker2    account


sudo docker-machine ip manager1       // get the ip address of manager

sudo docker swarm init -- advertise-addr  ipAddressOfManager1        // initilize docker swarm and assigns manager1  as manager role


sudo docker node ls     // tells manager1 is now assigned as manager role. this command will not work on worker node


sudo docker swarm    // shows all the commands

sudo docker swarm join-token worker1     // for manager to join the worker1


sudo docker swarm join-token manager1  // for worker1 to join the manager1


sudo docker info      // get details from manager1 account


   // create replica


sudo docker service ls   // check the status 


sudo docker service  ps web1  // 


sudo docker service scale web1=4  // scale service up and down


docker node inspect self


sudo docker node update --availability drain worker1Error: No such node: worker1 // shut down node


sudo docker swarm leave


Pending items: practical of all above commands and docker swarm 


*****************************************************************************

DOCKER ON WINDOWS PRO :

download "Docker desktop" and install 

c:\ docker  ==> check if docker is installed or not

c:\ docker version ==> check the docker version


download VNC viewer from https://www.realvnc.com/en/connect/download/viewer/

No comments:

Post a Comment