Basics | Ukant Jadia

A Beginner's Guide to DevOps with Terraform and GitHub

Mon, 30 Dec 2024 14:15:59 +0530

A Beginner’s Guide to DevOps with Terraform and GitHub

As a fresh graduate in Computer Science and Engineering, my exploration into the world of DevOps has been nothing short of an enlightening adventure. Coming from a machine learning background, I often found myself compelled to understand how backend systems could be crafted with such elegance. This curiosity fueled my capstone project, where I aimed to create a CI/CD pipeline that elegantly demonstrated the power of tools like Terraform and GitHub.

What is DevOps and Why is it Important?

DevOps blends development and operations, aiming to minimize the distance between these two traditionally siloed groups. The goal is to improve collaboration, increase deployment frequency, and deliver more stable operating environments. The DevOps culture promotes continuous integration and continuous delivery (CI/CD), which significantly enhances the software development lifecycle.

Key Components of DevOps

Continuous Integration (CI): Developers frequently integrate their changes in a shared repository.
Continuous Delivery (CD): Automating the delivery of applications to selected infrastructure environments.
Infrastructure as Code (IaC): Managing infrastructure through code, ensuring that setups are easy to replicate.

A Personal Exploration into DevOps

Reflecting on my academic journey, the decision to delve into a DevOps project stemmed from a fundamental question: How can vast complexities be integrated seamlessly? My project aimed to demystify this by constructing a basic CI/CD pipeline orchestrated through Terraform and GitHub.

Setting the Stage: Understanding My Project

The core of my project revolved around leveraging Terraform for infrastructure automation, along with GitHub for version control and CI/CD capabilities. This combination would provide a straightforward yet powerful example of deploying applications in a cloud environment.

Project Objectives

Demonstrate CI/CD using GitHub Actions: Show how automatic deployment can facilitate rapid product releases.
Use Terraform for Infrastructure Management: Illustrate how infrastructure can be provisioned and managed as code.
Visualize Workflows: Employ tools like Eraser.io to create flowcharts to help visualize the project’s structure.

Getting Started with Terraform

When I first explored Terraform, I was struck by how declarative it is. Terraform allows you to define your infrastructure in configuration files which can then be executed to provision that infrastructure automatically.

Example Terraform Configuration

Here’s a simple example of a Terraform configuration file that provisions an AWS EC2 instance:

provider ""aws"" {
 region = ""us-west-2""
}

resource ""aws_instance"" ""my_instance"" {
 ami = ""ami-0c55b159cbfafe1f0""
 instance_type = ""t2.micro""

 tags = {
 Name = ""MyInstance""
 }
}

Key Features of Terraform

Infrastructure as Code: Treat infrastructure setup as code, making it easier to maintain and version control.
Execution Plan: Terraform generates a clear execution plan which shows what will happen before any changes are applied.

Integrating GitHub Actions into the Pipeline

GitHub Actions provides a robust platform for automating workflows directly in your GitHub repository. For my project, I set up actions to automatically test and deploy the application upon code commit.

GitHub Actions Workflow Configuration

This YAML file illustrates a basic GitHub Actions workflow:

name: CI/CD Pipeline

on:
 push:
 branches: [ main ]

jobs:
 build:
 runs-on: ubuntu-latest
 steps:
 - name: Check out the code
 uses: actions/checkout@v2

 - name: Set up Node.js
 uses: actions/setup-node@v2
 with:
 node-version: '14'

 - name: Install dependencies
 run: npm install

 - name: Run tests
 run: npm test

 - name: Deploy to production
 run: npm run deploy

Benefits of Using GitHub Actions

Seamless Integration: Works directly with your repositories.
Customizability: Create tailored workflows to fit specific project needs.

Visualizing the Workflow

To visualize how data flows through the different stages of the pipeline, I employed Eraser.io. This tool allowed me to create flowcharts that represented the overall CI/CD process from code push to deployment.

Figure 1: The flow of code and data in the CI/CD pipeline

The Elegance of CI/CD: A Real-World Application

As I progressed through my project, witnessing the pipeline’s operation became one of the most rewarding aspects. It was astounding to see how a simple workflow could articulate the path of requests or data between containers and servers.

Key Takeaways from the CI/CD Pipeline Experience

Visual Learning: The flowcharts provided invaluable insight into how requests and data moved, making the whole process comprehensible.
Hands-on with Real Tools: Working directly with GitHub and Terraform solidified my understanding of cloud infrastructure and automation.

Final Thoughts

Reflecting on this project, I can confidently say that building a CI/CD pipeline using Terraform and GitHub not only quenched my curiosity about backend processes but also equipped me with practical skills for my career ahead. This journey reaffirmed my belief that technology can create elegant solutions to complex problems. I hope my exploration encourages others to dive into the DevOps ocean, where every wave brings new learning opportunities.

References

Feel free to explore these resources further to expand your knowledge and understanding of DevOps and CI/CD pipelines.

What I Know About Java

Sat, 30 Dec 2023 14:23:50 +0530

Explaining the

pubic static void main(String[] args)

the execution of our code starts form main.
let in order to create the object of class x we should have the main method
So main is a parameter and it takes parameter ie. main ( )
And the type of the parameter can be anything so we take the string because String except values and we don’t know how many values we are taking so we take array main(String args[])
There is nothing to return in main so it is a void type
In order to call the method we require the object and the execution of our code is start from the main so that’s why it considered as static void main(String args[])
and it made public so public static void main(String args[])

creating a object inside the main

we type the class name and obj name Main obj
then we have a keyword new : it tell the jvm how memory it required for the given class, for that the size of class is define by the default constructor.

it is like a blowing the air inside the pillow (keyword new ), and capacity of that pillow is constructor.

Java OOP

Method Overloading

Methods with same name but diff in (parameter or data type of the(both)).
this kind of methods are known as overloaded method. ex :

void func(){..}
int func(){...}
float func(int x){...}
void func(int x, float k, long l){...}

Method overloading should be consider by the type or count of the parameter that passed to a method not by the return type of the method.

Constructor

It is member method.

Constructor have same name as class name.

Constructor will never return anything.

It used to allocate the memory.

Every time when we have to create a any object we have to use the constructor weather we define it or not.

If we don’t create a constructor it will be there that’s a default constructor.

What a Constructor do : it will define the size of class which help keyword new to create the memory accordingly .

unlike methods, a constructor has the same name as that of class and does not have any return type.
In java similar to method but only invoked when an object of the class is created. ex:

class Main{
 private short num;

 // constructor
 Main() {
 System.out.println("After the object is create. ");
 num = 27;
 }

 public static void main(String[] args){
 // object of class is created 
 Main obj = new Main();
 // calling the var
 System.out.println("the short number is "+ obj.num);
 }
}

There are three type of Constructor
Private Constructor : Private constructor can’t be accessed from outside the class.
No-Arg Constructor : Constructor that does not accept any arguments.
Parameterized Constructor : Constructor that accepts arguments.
Default Constructor : Constructor that is automatically created by the Java compiler if it is not defined.

A constructor can not be abstract, static or final

Access Modifier

It is helpful in the Encapsulation ( preventing the data from misuse and alter.).
And to Control what part of program can access the members of a class.
Default : Declarations are visible only within the package(package private).
Public : This can be access by other class.
Private : This can not be accessed by other class.
Protected : Declarations are visible within the package or all subclasses.
refer to this image this

this key word

this : refer to the current object inside a method or constructor.
in other words: this is treating the variable with same name, differently according to there position/place.
using this with getters and setters : in getters/setters we are using the same var name as in the program
using this in constructor overloading ex:

class Complex {

 private int a, b;

 // constructor with 2 parameters
 private Complex( int i, int j ){
 this.a = i;
 this.b = j;
 }

 // constructor with single parameter
 private Complex(int i){
 // invokes the constructor with 2 parameters
 this(i, i);
 }

 // constructor with no parameter
 private Complex(){
 // invokes the constructor with single parameter
 this(0);
 }

 @Override
 public String toString(){
 return this.a + " + " + this.b + "i";
 }

 public static void main( String[] args ) {

 // creating object of Complex class
 // calls the constructor with 2 parameters
 Complex c1 = new Complex(2, 3);

 // calls the constructor with a single parameter
 Complex c2 = new Complex(3);

 // calls the constructor with no parameters
 Complex c3 = new Complex();

 // print objects
 System.out.println(c1);
 System.out.println(c2);
 System.out.println(c3);
 }
}

passing this as an argument : - i also didn’t get it. ask to sir

final keyword

final keyword means used to denote the constants and it can be used with variables,method and classes.
final variable cannot be reinitialized with another value.
the final method cannot be overridden.
the final class cannot be extended.

`instanceOf` operator

used to check whether an object is an instance of class or not
syntax objectName instanceOf className;

Inheritance

It allows to create a new class from an existing class.
just inherit it from it’s father class.

class Main
{
 public static void main(String args[])
 {
 child boy = new child();

 boy.display();
 }
}

class parent{
 String color = "White";
 System.out.println("It is same like we get the traits from our parents, JDK or JVM also doing the same thing.");
 // it is same like we get the traits from our parents, JDK/JVM/java is also doing the same thing. 
}


// inherit from parents
class child extends parent{
 public void display(){
 System.out.println("The value for variable \'color\' is comming from the parent class.");
 System.out.println("Color of child is "+color);
 }
}

Network Concepts Basics

Sat, 30 Dec 2023 14:21:26 +0530

Networking Concept

Mac address

Media control access
It is physical address to identify the devices
it identify the NIC of each divices (NIC–> Network identification card)
ex:- ether 00:0c:49:0a:43:03
Mac work in the second and third layer of the OSI model
Specially it is related to the switches

TCP –VS– UDP

TCP

Transmission COntrol Protocol
Connection oriented protocol
used in HTTPS/SSH/TelNet/SMTP/FTP
provide the more security and relailablity
it form one-to-one connection and it gurantee the complete delivery of data
Three way hand shake
SYN -> SYN-ACK -> ACK
it is slow due to three-way-handshake, it take time to complete and establis the connection
Use port to establish the connection
During hand-shake it acknowldge your machine to the requested server
contains the 20Byte packet

UDP

User Datagram Protocol
Connection less Protocol
it provide the fast transmission
Broadcast or multicast the message and the complete delivery of data is note guarnted
Streamming on voice-over-io(VOIP), DHCP,DNS
Used for streaming the data like procast, video
contains the 8Byte packet

Comman Ports and Protocol

TCP

 PROTOCOL/SERVICE PORT'S
FTP 21
SSH 22
TelNet 23
SMTP 25
DNS 53
HTTP/HTTPS 80/443
POP3 110
SMB 139+445
IMAP 143

UDP

 PROTOCOL/SERVICE PORT'S
DNS 53
DHCP 67.68
TFTP 69
SNMP 161

The OSI Model

Open Systems Interconnection(OSI)
Learn this mnemonic to to know the seven layer sequence
Please Do Not Throw Sausage Pizza Away

OSI seven layers are

 1. Physical (P)Please
2. Data link (D)Do
3. Network (N)Not
4. Transport (T)Throw
5. Session (S)Sausage
6. Presentation (P)Pizza
7. Application (A)Away

OSI Model Explain REF

 Layers For
1. Physical -- Provide the connection between computer and network, it includes physical datacable,server
2. Data Link -- Make sure the data-transfer is error-free between different node, when data packet arrive at network layer then data link layer(DLL) deliver that packet to the requested host using MAC address.
3. Network -- Transmitt the data from one host to other, in different netwroks.It also place the reciver/sender ip address in header of packet.
4. Transport -- Data packet referred as Segment, it is responsible for the end-to-end delivery of complete message without any error/packet loss.
5. Session -- Responsible for the establishing the session,maintaining and also ensure the security
6. Presentation --- Data from application layer(layer 7) is extracted here and manipulate as per the required to transmit over network.
7. Application --- Used by the end user software such as web browser and email clients.

While Receiving the data from network model work from layer 1 –> layer 7

While sending the data from machine model work from layer 7 –> layer 1

Networks

IP address are work with the third layer of the OSI Model
There are two types of ip address
- IPv4 : 192.168.34.123
- IPv6 : 23:s3:3w:34:dw:23:2d
In IPv4 each section after delimeter(.) is contain the 8 bits each (0 & 1)
The total of complete ip address is 32 byte.
the total in terms of the bits is 256 that is from 0 to 255.
There are many network around you and your system and all are having the very specific identity number that is known as IP address.
Types of ip address in a network

Class Start ip End ip Subnet Mask Netwroks(max) Hosts(max)

A 0 127 255.0.0.0 126 16,646,144

B 127 191 255.255.0.0 16,383 15,024

C 192 223 255.225.255.0 2097,151 254
IP address in the binary format
- 11111111.11111111.11111111.0 == 255.255.255.0
- == 192.168.15.
Some Standard Slash Network
- /24 Network
  
  128 64 32 16 8 4 2 1
  
  1 0 0 0 0 0 0 0
- /25 Network
  
  128 64 32 16 8 4 2 1
  
  1 1 0 0 0 0 0 0

Class	Start ip	End ip	Subnet Mask	Netwroks(max)	Hosts(max)
A	0	127	255.0.0.0	126	16,646,144
B	127	191	255.255.0.0	16,383	15,024
C	192	223	255.225.255.0	2097,151	254

128	64	32	16	8	4	2	1
1	0	0	0	0	0	0	0

128	64	32	16	8	4	2	1
1	1	0	0	0	0	0	0

Links

It is getting hard for me to write and draw the ahed so i am adding some link of my document go through them, Some link to internet resources

Information Gathering, the First Step for Exploitation

Sat, 30 Dec 2023 14:20:17 +0530

Recon

There are two type of Reconnaissance

Passive Reconnaissance :– Doing the recon without coming in direct contact of the target. \
Active Reconnaissance :– Doing the active notecible action on the target site.

Passive Reconnaissance includes

local/Geo Information
- Satellite Images
- Building layout
- Cabin location, fence, count of head in building, ways to exit, break areas, security fencing.
Job information
Senior manager name, meeting routine, job title.
Image from social sites like badge photo, cubical, desktop.

Web and Network Information

Targe Validation

First thing you have to do is, validate your target.

Weather the description of given target is having the relating with the given ip

Finding Subdomain

Testing only one domain or some domain is decrease the chances of finding the flaw.

Testing the all in-scope sub-domain is a good practice.

Tool can be use for finding the subdomain are

Google fu, dig, Nmap, sublist3r, bluto, crt.sh etc.

Fingerprinting

We need to know which website running what on front or back.

What’s running on the host.

What’s web server, ssh, ftp, services.

Data Breaches

Just check if the credentials or account is pawned or not

Use this to check

havebeenpawned, Breach-prose, weleakinfo.

NOTE: Better Recon, enumeration makes you better pentester.

Email Address gathering with Hunter.io

You can use Hunter.io to find the people email address working or related to the domain you enter.
In free plan it only provide the 20 searches as free, so do not abuse the searches.
It also tell the where it get all emails written in source.
You have to login with valid email id to have account.
It also provide the category of email address, the department of the email owner.

Breach-Phrase from github

It is tool on the GitHub, it checks for the leaked credentials in its own in-build data of credentials
This in build data is gather from the dark web store.

NOTE: the gather email address are plays important role in further recon and enumeration.

Utilizing the Harvester

It searches with the domain.
It can gather the subdomain name, e-mail addresses, virtual hosts, open ports/banner, and employee name from different public sources like search engine, social site

Hunting Subdomain

We need see/identify what are the available or working subdomain out there. use this

Sublist3r
- Sometimes it only give the 3 level subdomain
crt.sh
- It give the all registered subdomain that can be 3 level or 4 level
OWASP Amass is more efficient tool than above mentioned
Sublist3r is slow during to fast up increase the threads NOTE: Found subdomain may or may not alive.

Identifying the Web Technologies

To Fingerprinting the web page or to get the services running in the background
wappalyzer is browser extension which show the frontend and backend services.
Terminal tool is whatweb
- use whatweb -h for more details of tool

Basic Git Commands

Sat, 30 Dec 2023 14:15:59 +0530

what you should do in git

git init -b main
git remote add origin <repo_link>
git remote -v

cd to the root file of the project
echo "+ Project " >> README.md
git init
git add .
git commit -m "README"
git branch -M Main
git remote add origin <repo_link>
git push -u origin Main

pushing an existing repo from cli git remote add origin https://github.com/UserCtf777/Inkdrop.git git branch -M Main git push -u origin Main

after clonning any repo it have the address of origin, simply you cannot add your repo url for backup/project/ICR for that you have to change the origin url follow the following command

to check the origin url try the remote verbose git remote -v
to change the remote origin try git remote set-url origin <repo_url>

new error found
sometimes when we clone the repo we use depth=1 to remove it’s history
this give the error during pushing it to other origin
error looks like

 ![remote rejected] Main -> Main (shallow update not allowed)

```
 error: failed to push some refs to
```
to solve this try git remote set-url <old_url> git fetch –unshallow old

git init -b main
git remote add origin <repo_link>
git remote -v

cd to the root file of the project echo “+ Project " » README.md git init git add . git commit -m “README” git branch -M Main git remote add origin <repo_link> git push -u origin Main

pushing an existing repo from cli git remote add origin https://github.com/UserCtf777/Inkdrop.git git branch -M Main git push -u origin Main