Introduction to Python

Ok, let´s start coding with these new bits of info we already have.

x = 5

y = 4

print(x < y)

Result = FALSE

x = 5

y = 4

print(x > y)

Result = TRUE

x = 5

y = 4

print(x >= y and y == 4)

Result = TRUE

import random

x = random.randint (1, 5)

if x == 1:

print ("I am number 1")

elif x == 2:

print ("I am number 2")

else:

print ("I am number", x)

if we get 1 -> I am number 1

if we get 2-> I am number 1

if we get another number -> I am number 1

For loop

import random
y = 0
for i in range (10):
	y += random.randint(1, 5)
	print (y)
print ("y = ", y)

Result:

While

import random
while w != 5:
	w = random.randint (1, 100)
	print  (w)
print ("\nw = ", w)

Lists

Lists are like variables BUT they can store several ordered values

This is a LIST -> [5, 4, 24, 36, 9]

But we can also have a LIST with different elements, like numbers, float, strings…

The awesome feature of LISTS is that being an ORDERED structure. EACH element within the list has an INDEX (starting from 0), so we can call each element of the list with no mistake.

Take a look at the previous image; starting from the right, we get the FIRST index as 0; starting from the left, e have NEGATIVE indexes, in this case, we have -3.(starting with -1)

To create a LIST, we must put a name = [] -> being the [] what defines our list.

list_! = [] -> this is an empty list

list_2 = [2, 5, 9, 6, 56] #-> this is a FIVE elements list (always comma separated)

So, let´s say I want to access the third element of this list. How can I do that? Well, remember that a list has ORDERED elements with INDEXES starting from 0 (right) or -1 (left), so I can call it this way:

list = [4, 6, 78, 43, 56]

print (list[2]) #-> it will show 78 (index0, index1, index2 - third position)

Now, let´s say I want to access the last element ...we can use negative indexes (remember that negative indexes start with -1), so the last element would be

list = [4, 6, 78, 43, 56]

print (list[-1]) #-> it will return 56

We can also ADD (append) elements to a list.

Appends ADDS an element at the end of the list

list = [4, 6, 78, 43, 56]

list.append (21)

print (list)

[4, 6, 78, 43, 56, 21]

We can also REMOVE an element from the list, by VALUE (bear in mind that it will remove the first occurrence from the list)

list = [4, 6, 78, 43, 56]

list.append (21)
list.remove (6)
 #-> it will remove the VALUE 6
print (list)
[4, 78, 43, 56, 21]

Or we can remove a value by INDEX using POP

list = [4, 6, 78, 43, 56]
list.pop (4)
#-> it will remove index 4, fifth position; number 56

print (list)
[4, 6, 78, 43]

We can aslo EDIT or REPLACE elements within the list

list = [4, 6, 78, 43, 56]

list [0] = 456# replaces index 0, first element (4) with 456
print (list)
[456, 6, 78, 43, 56]

list = [4, 6, 78, 43, 56]


list [0] += 456# edit, adds 456 to first element
print (list)
[460, 6, 78, 43, 56]

How many elements has my list?

list = [4, 6, 78, 43, 56]

print (len(list))
# Result = 5

Show the value calling it by its index

list = [4, 6, 78, 43, 56]

print (list.index(78))

#Result = 2 -> index number 2 has the element/value 78

To check if a value is within a list

list = [4, 6, 78, 43, 56]

print (4 in list)

#Result = True

Sorting and reversing a list

list = [4, 6, 78, 43, 56]

list.sort() #shows list in ASCENDING order

print (list)

list.reverse()

print (list)

Result –> sort = [4, 6, 43, 56, 78]

Result-> reverse = [78, 56, 43, 6, 4]

Loop through a list

We use a FOR loop, setting a variable name to identify the elements within the list

list = [4, 6, 78, 43, 56]

For element in list:

print (element) -> for each loop, the element variable will take every value within the list

Result =

I can also loop the list by its index numbers instead of by its value

list = [4, 6, 78, 43, 56]

for i in range (len(list)): -> I = variable to host index, len to get the length of list

print (i)

Result =

I can also loop the list by its index numbers instead of by its value

list = [4, 6, 78, 43, 56]

for i in range (len(list)): -> I = variable to host index, len to get the length of list

print (i)

Result =

Finally I can loop the list by its index numbers bringing their values

list = [4, 6, 78, 43, 56]

for i in range (len(list)):

print (list [i])

Result =

Loop Brief

list = [4, 6, 78, 43, 56]

for element in list:

print (element)# loop by value

print ("\n")#adds a newline, a separator

for i in range (len(list)):

print (list [i])#loop by index

The two pieces of code return the same value

Strings

Strings can also be seen as Lists, they also have indexes

We can also count the elements of the string

string_test = "Hello World"
print (len(string_test))#we get the lenght of this string
Result = 11

We can also index a string like a list

string_test = "Hello World"
print (string_test[0])# I call the FIRST character of the string, index 0
Result = H

And we can loop the string

string_test = "Hello World"
for character in string_test:
    print (character)

We also have several functions to use with string

string_test = "Hello World"
print (string_test.lower())# convert text to lowercase
print ("\n")
print (string_test.upper())# convert text to uppercase
print ("\n")
print (string_test.capitalize())# capitalize text

hello world

HELLO WORLD

Hello world

We also have methods that return True or False

string_test = "Hello World"
print (string_test.startswith("Hi"))# result-> False
print ("\n")
print (string_test.startswith("He"))# result-> True
print ("\n")
print (string_test.endswith("wo"))# result-> False
print ("\n")
print (string_test.endswith("ld"))# result-> True
print ("\n")
print (string_test.isalpha())#return False because the space betweeno and w is not an alphabet character
print ("\n")
print (string_test.isdigit())# result-> False

Strip, to remove spces before and after text

string_test = " Hello World "#Note that we added a space before H and another after d

print (string_test.strip())Astrip will remove both spaces before and after text

Result
Hello World (with NO spaces before or after text

Split

Split will transform a string into a list, taking as separator the one we pass the function as parameter (usually a comma)

fruits = "apple, orange, banana, tangerine"# note that every fruit is comma separated
print (fruits.split(","))#split will return as a LIST, the COMMA is the separator

Result = ['apple', ' orange', ' banana', ' tangerine']

Join-> list to string

To use join the sintax is like this

“”.join, where “here goes the character I want to use to join”

fruits = "apple, orange, banana, tangerine"# note that every fruit is comma separated
list1 = fruits.split(",")#save the result to the variable list1

list2 = "-".join(list1)# the character "-" will join the list
print (list2)
Result = apple- orange- banana- tangerine

It goes from a list to a string joined by the “-” character

Note -> “handmade ” way to fill a List with characters from a string

Let´s say I have my string and want to “fill” a List with every character with a For loop. We can do it this way

fruits = "apple, orange, banana, tangerine"# note that every fruit is comma separated
print (fruits)
#prints the string fruits
print ("\n")

liststring = []
for c in fruits:
    liststring.append(c)
print (liststring)#prints the List liststring

Result = 
apple, orange, banana, tangerine


['a', 'p', 'p', 'l', 'e', ',', ' ', 'o', 'r', 'a', 'n', 'g', 'e', ',', ' ', 'b', 'a', 'n', 'a', 'n', 'a', ',', ' ', 't', 'a', 'n', 'g', 'e', 'r', 'i', 'n', 'e']

Ok, of course, that if you want to append the words (NOT the characters one-by-one) there is always an easy and right way to do it

fruits = "apple, orange, banana, tangerine"
lista = []
lista.append(fruits)
print (lista)

Result = ['apple, orange, banana, tangerine']

or if you want to convert a string to a list just put the string name within []

fruits = "apple, orange, banana, tangerine"
lista = [fruits]
print (lista)

Result = ['apple, orange, banana, tangerine']

Tuple-> immutable Lists; once defined we can not change it.

It is defined as a List, BUT with () instead of []

tupla1 = (1, 3, 5)

print (tupla1, "\n")

Result = (1, 3, 5)

I can access its indexes/values like a List

tupla1 = (1, 3, 5)

print (tupla1 [0], "\n")

Result = 1

We can make some operations with the tuple.

tupla1 = (1, 3, 5)

print (tupla1[0] + tupla1[1])# we´ll get 4 (1 + 3)

print (tupla1)

tupla2 = tupla1 + (1, 3, 5, 8)
# we add tupla1 plus values 1, 3, 5 and 8
print (tupla2)

Result = 
4
(1, 3, 5)
(1, 3, 5, 1, 3, 5, 8)

BUT now if i want to edit it I can´t do it, because it is an immutable List

var = (3, 4 , 8)
var[0]=8
print (var)
Result = 
Traceback (most recent call last):
     var[0]=8
TypeError: 'tuple' object does not support item assignment

I can also convert a List to a Tuple, using the reserved word TUPLE

Take a look at the example, we also show the TYPE of the variables to check they are a List or a Tuple

var = [3, 4 , 8]
print("var = " ,type(var))
tupla = tuple(var)
print (var)

print("\n")
print("tupla = " ,type(tupla))
print(tupla)

Result = 
var =  <class 'list'>
[3, 4, 8]

tupla =  <class 'tuple'>
(3, 4, 8)

TIP: Tuples are Faster than Lists

SETS

Sets DO NOT keep an order (they have no index), the go between curly braces {}

They don´t allow duplicated elements, so we can tell that a SET is a List of UNIQUE and UNORDERED elements

As any Set, we can perform some operations with them like union, intersection, difference and symmetric difference.

we have two ways to create a set; by assigning a variable to a {} with comma-separated elements within

set_1 = {1, 4, 6, 8,55}

or take a List and convert it to a set with the reserved keyword SET

list_1 = [5, 4, 76, 22]
set_2 = set(list_1)

We can add elements, but with the ADD function (APPEND is for Lists)

set_1.add(10)

Now let´s see what happens when we want to add a duplicated element

set_1 = {1, 4, 6, 8, 22}
print (set_1)

Result = {1, 4, 6, 8, 22}
#now we add a duplicated element
set_1.add(22)
print (set_1)

Result = {1, 4, 6, 8, 22}

as we can see, we have NO error but NO change, the SET just ignores the duplicated element and returns the same set with no change

We can remove an element

set_1.remove(22)
print (set_1)

Result = {1, 4, 6, 8}

We can also loop the set

set_1 = {1, 4, 6, 8}
for elem in set_1:
    print (elem)

Result = 
1
4
6
8

Ok, let´s check if a given element is within the set with the IN keyword

set_1 = {1, 4, 6, 8,22}
print (5 in set_1)

Result = False -> there is NO 5 in our set

set_1 = {1, 4, 6, 8,22}
print (6 in set_1)

Result = True -> yes, we have a 6

Tip: searching within a Set is fastest than doing it within a List

Let´s test some operations with sets

Union

set_1 = {1, 4, 6, 8,22}
set_2 = {11, 2, 6, 8, 9}

print (set_1.union(set_2))

Result = {1, 2, 4, 6, 8, 9, 11, 22}
 -> note that we had duplicated 6 and 8 and the final set did NOT duplicate them, just kept one element

Intersection

set_1 = {1, 4, 6, 8,22}
set_2 = {11, 2, 6, 8, 9}
print (set_1.intersection(set_2))

Result = 
{8, 6}

Dictionaries

Dictionaries DO NOT store individual values but key-values pairs

dic1 = {"name": "Freelancer", "age": 45, "profession": "developer"}
print (dic1)
Result = {'name': 'Freelamcer', 'age': 45, 'profession': 'developer'}

Note the example; we have 3 key-values elements (comma-separated) with the first key-element “name” and value 19, and so…

we can think of dictionaries as unordered lists (we have no indexes) with no content restriction (I can “mix” numbers and letters)

Either way we can access any element within the dictionary because they have a key to every element.

So the keys would be our indexes -> I get the values through the keys

A good practice is to write dictionaries this way (pretty much like Json code) to make them clear

dic1 = {"name": "Freelancer",
        "age": 45,
        "profession": "developer"
        }
print (dic1["age"])#we call the value (45) by its key (age)

Result = 45

Adding values

dic1 = {"name": "Freelancer",
        "age": 45,
        "profession": "developer"
        }
dic1["country"] = "Argentina"# add a new key-value pair
print (dic1)

Result = {'name': 'Freelancer', 'age': 45, 'profession': 'developer', 'country': 'Argentina'}

Deleting and editing values

dic1 = {"name": "Freelancer",
        "age": 45,
        "profession": "developer"
        }
dic1["country"] = "Argentina"# add a new key-value pair
del dic1["age"]# delete a pair
dic1["name"] = "Web Developer"#edit a value
print (dic1)

Result = {'name': 'Web Developer', 'profession': 'developer', 'country': 'Argentina'}

Looping the dictionary

Ok, we´ll use for as usual, BUT now we have to go though 2 values, not just one (for element in….working no more), so how can we do that?

for key_variable_name, value_variable_name in list (dictionary_name.items()):

dic1 = {"name": "Freelancer",
        "age": 45,
        "profession": "developer"
        }
for k, v in list(dic1.items()):
    print (k, v)

Result = 
name Freelancer
age 45
profession developer

Nested structures

dic1 = {"name": ["This is a list within a dictionary"],
        #and now another dictionary within the dic1
        "details": {
            "age": 45,
            "profession": "developer",
            "country": "Norway"
        }
    }
for k, v in list(dic1.items()):
    print (k, v)
Result = 
name ['This is a list within a dictionary']
details {'age': 45, 'profession': 'developer', 'country': 'Norway'}

But, how do I access the age? I should “index” from the outer structure to inner one

dic1 = {"name": ["This is a list within a dictionary"],
        #and now another dictionary within the dic1
        "details": {
            "age": 45,
            "profession": "developer",
            "country": "Norway"
        }
    }
print (dic1["details"]["age"])# note the double "indexation"
#[details][age] to finally reach the value

Result = 45

Another example

dic1 = {"name": ["This is a list within a dictionary", 89, 65, 77],
        #and now another dictionary within the dic1
        "details": {
            "age": 45,
            "profession": "developer",
            "country": "Norway"
        }
    }
print (dic1["name"][1])# note the double "indexation"
#[name][1] to finally reach the value

Result = 89

Tip: accesing a value within a dictionary is faster than a list

Functions

A function is a block of code with a anme associated to it, which only runs when it is called.

You can pass data, known as parameters, into a function.

A function can return data or execute a task as a result.

Bear in mind that everything used so far are functions (print, len, int, range, input and so). Here a list of built-in Python functions

But we can also create OUR OWN functions. Remember that we can call a function several ways, that´s one of its main goals; doing a repetitive task several times without the need to coding all again and again, just call the function wiy¿th the new parameters and we are done.

Let´s say I want to get the summation of all numbers within the list, and that I am gonna use this 5 times in my program. I can write 5 times the code to do the same task, or I can create a function and call it several times without the need to code all from scratch over and over again.

We start by creating our new function with the reserved word “def” followed by the “name_of_this_function” and “()“. We finish the command with “:”

Inside the () we must pass the “parameters” for the function to work with. The parameters are all the information that the function needs to operate. Usually we{ll work with some kind of abstraction, meaning, let´s say I want to create the function to add all the elements of a list, so my main parameter will we a List, then I should create a function like this

def myfunct (list_1):

Where the (list_1) parameter is not yet created, BUT we need a list to work with, the function needs a parameter

Ok, once created the function, and after the “:” note that the next line will be indented, we should define the function as it, this menas, we created it with “def”, now let´s define what the function should do.

Ok, we wanted to sum up all the elements of the list, so it would be nice to initialize a variable (x = 0 ) to keep tracking of the summation of elements from the list

And then we should do a for loop to go through all the elements in the list, as usual, AND sum them.

 x = 0
    for elem in list_1:
        x += elem

Now we have the sum of tle elements in the list stored within the “x” variable, so our function worked so far, but the function not only receives parameters, but it also returns a result, so how can we get that result? Of course, with the reserved keyword “return”. And, what should we return? well, in this case the “x” variable containing the sum of elements

Note that so far it is all an abstraction, nothing happens yet, if I do run this code nothing seems to happen

def myfunct (list_1):
    x = 0
    for elem in list_1:
        x += elem
    return x
Result = 
>>> %Run functions.py

To use this function, we need to “call” it. BUT to call it we have to satisfy the parameters, meaning, I have to “pass the parameters” to the function, so it can execute the abstraction and returns a value

How do we call a function? Well, by its name -> myfunct

BUT we must save the results sent to myfunt in a variable so we can print it, AND we must pass the list (between []) to the function.

summation = myfunct ([1, 2, 3, 4, 5, 6, 7, 8, 9])
print (summation)

Remember that we did this?

def myfunct (list_1):

Well, this one

myfunct ([1, 2, 3, 4, 5, 6, 7, 8, 9])

“fills” (list_1) with the elements that are used by the function. It sounds a bit weird, I know, is like starting from the end, but that´s how it works. It happened why when we “call” the function with the parameters, then it is executed the “abstraction” of the function, not before, this is not line-by-line from 1 to 10, but it goes from start to the end, fetch the parameters and then executes the code and show the results.

Let´s see and check the full code


def myfunct (list_1):
    x = 0
    for elem in list_1:
        x += elem
    return x

summation = myfunct ([1, 2, 3, 4, 5, 6, 7, 8, 9])
print (summation)

Result = 45

So, when I call the function by its name (myfunct) it is the moment the function runs using the values that I pass as a List (note that we assign the function to a variable “summation”so we can later print it)

So, the parameters are the communication between the program and the function, and the return is the communication between the function and the program.

We must pay attention to variables when defining functions; local and global ones.

In our recent example we defined the variable as “local” to the function, meanind that if we call the variable “out” of it, we´ll get an error . Let´s check

def myfunct (list_1):
    x = 0
    for elem in list_1:
        x += elem
    return x

print (x)-> calling a local variable out of the function
summation = myfunct ([1, 2, 3, 4, 5, 6, 7, 8, 9])
print (summation)

Result = 
print (x)
NameError: name 'x' is not defined

We get this error because the “x” variable is defined WITHIN the function. The same thing will occur if I define a variable out of the function…my function will not recognize it because it is “out” of the function environment -> myfunct do not know the variable “summation”

Then, we´ll repeat thi “The communication between the function and the program is through to the parameters, this info (parameters) I give to the function are used to get a result, how can I get those results? with a return”

I can also use default parameters, this is, if the code do not pass me parameters, the function uses its own definde default parameters

Classes and Objects

Here a full doc

To understand the difference between Classes and Objets we´ll get a real nice example; let´s suppose you are an architect building some houses.

In order to build them, you FIRST need drawings/designs regarding how the house will look, with its pipes, windows, floors, and more.

ONCE you have those plans, THEN you build the house or houses.

Then:

Plans/drawings -> CLASSES

Houses/Buildings -> OBJECTS

Classes are just concepts (abstractions) in where we initialize/create the Objects
Furthermore, to every class-plans, I can define multiple PROPERTIES (color, power consumption, and so) that could change to every object-house.
Finally, I can add some METHODS to the classes defined, so my Objects can perform or be able to do some action (maybe ring a bell in a house)

So we end with this:

Plans/drawings -> CLASSES

Houses/Buildings -> OBJECTS

Attributes/Features -> PROPERTIES

Actions -> METHODS (functions that work WITHIN the Class, and can only be used but the Objects of the class)

Ok, enough theory let´s create a class

We have a reserverd word for Classes called…you guessed: class. Then the name of our abstraction/class followed by “:”

After that we need to use a “constructor; “__init__” is a reseved method in python classes. This method is called when an object is created from a class and it allows the class to initialize the attributes of the class

“The constructor allows us to add the properties we wish for our “house” (object)

Remember that a Method is just a function that work ONLY within the class for the objects created for that class, so you guessed again, an __init__ method is defined with….def.

def __init__ (self, parameters) -> remember that we always get the “self parameter”

def __init__ (self, color)

Then, to define the properties of my Object (house), I have to use self for every one of them. Self works as a dictionaty (pair key-value) BUT using this sintaxis; self.property = value (self.color = color) -> that color variable is this one def __init__ (self, color)

So far we have:to build a house, assign it a color, we have 0 water and electricity consumption, so the code is

class house:
    def __init__(self, color):
        self.color = color
        self.electricity = 0
        self.water = 0

Then, EVERY house I build, will have 3 properties; color, electricity and water consumption.

These “self” are very important because we can use them to access these properties from within another methods

Now we could define the methods; what can I do with a house? Maybe we could paint it, so let´s define a paint method

Remember that every method within a class MUST have the self as first parameter

def paint (self, color):
        self.color = color

Now we can define another methods like…using the lights,

def lights_on (self):
        self.electricity += 10

the water

 def use_water(self):
        self.water += 7

and the doorbell

def ring_doorbell(self):
        print ("rinnnnnnnnnggggggg!!!!!!")
        self.electricity += 1

Our code so far would look like this

class house:
    def __init__(self, color):
        self.color = color
        self.electricity = 0
        self.water = 0
        
    def paint (self, color):
        self.color = color
        
    
    def lights_on (self):
        self.electricity += 10
        
    def use_water(self):
        self.water += 7
        
    def ring_doorbell(self):
        print ("rinnnnnnnnnggggggg!!!!!!")
        self.electricity += 1

Bear in mind that if we run this code NOTHING happens (or shoe) because we just DEFINED the Class, we only have the plans, we did not put a single brick of our house yet

Let´s begin to create our objects

First I choose a variable to host the object, the class name and the parameters to “build the house” in this case “color” that we created before in the constructor (__init__). Remember that in our example we defined only one parameter; color (electricity and water are initialized to zero, not passed as parameter)

So we “call” the constructor method with the parameters defined in the method

my_house = house ("red")
 -> we call the constructor method to build the house with a red color

So this piece of code will build a RED house with ZERO water and electricity consumption

class house:
    def __init__(self, color):
        self.color = color
        self.electricity = 0
        self.water = 0

Now that the object is created, how do I access it? How can I see it?

It is as easy as invoking the variable name where we host the object DOT the property name

print (my_house.color)
print (my_house.electricity)
print (my_house.water)

Result = 
red
0
0

Now I can start calling the METHODS we created for our Object, so we´ll try with the doorbell

my_house.ring_doorbell()
print (my_house.electricity)

Result =
rinnnnnnnnnggggggg!!!!!!
1 -> note that before we had 0 consumption, but now we have 1 because we called the method my_house.ring_doorbell()
 and it performed the action +=1

Now I can paint again my house calling the paint methos BUT passing a different parameter-color

my_house.paint ("green")
print (my_house.color)

Result = green

The full code so far with comments so it is easy to understand:

class house:
    #I create the class "house"
    def __init__(self, color): #I invoke the constructor with only one parameter; color
        self.color = color #property color
        self.electricity = 0 #property electricity
        self.water = 0 #property water
    #so we have or class-plans created with the properties color, electricity and water
     
    #Now we start creating methods 
    def paint (self, color):#method paint
        self.color = color
            
    def lights_on (self):#method lights_on taht will add 10 to the 0 initial state
        self.electricity += 10
        
    def use_water(self):#method use_water taht will add 7 to the 0 initial state
        self.water += 7
        
    def ring_doorbell(self):#method ring_doorbell that will print a mesaage and add 1 to the electricity state
        print ("rinnnnnnnnnggggggg!!!!!!")
        self.electricity += 1
#so fr we have only created the plans, all abstraction        

#now let´s create the object house
#I create a variable (my_house) who will host the object house with a
#color parameter (red)        
my_house = house ("red")

print (my_house.color)#shows the house color
print (my_house.electricity)#show electricity use
print (my_house.water)#shows water use

my_house.ring_doorbell()#we call the methos ring_doorbell
print (my_house.electricity)#we show the electricity use, notice now is 1 and not 0

my_house.paint ("green")#we call the paint methos and our house change from red to green
print (my_house.color)

Result = 
red
 
0
0
rinnnnnnnnnggggggg!!!!!!
1
green

Brief: the Class is the abstraction, so I began to build it when I call the class and pass it parameters so the constructor method can build the object.

But, what if I want to use the rpevious class-plans to build another thing, not a house but a mansion maybe?

Of course, I could write the class-plans from scratch, but it is wiser to get the useful things from our class and use them, this is called inheritance -> is the mechanism of deriving new classes from existing ones

The new (child) class will have the same properties and method than the father class

And how do we get this done? Just by creatinbg a new class and, between brackets, the name of the father -class

class mansion (house):

So our mansion class will have the same properties and methods as “house”, but the reason to do this is to change some methods as we need them to perfom another actions.

Let´s say that I want to change poweer consumption , so my mansion will spend more than 10 of electricity

class mansion (house):
    def lights_on (self):
        self.electricity += 38

Note that we did not use a constructor (__init__) because it is inherited from the father (house)

We can also modify the other methods too

class mansion (house):
    def lights_on (self):
        self.electricity += 38
        
    def use_water (self):
        self.use_water += 19
    
    def ring_doorble (self):
        print ("Ding-Dong!!")
        self.ring_doorbel += 3

So we changed 3 methods,

Now to create the objet, let´s assign it to a variable as we did before, the name of the class and between brackets the parameter to the constructor (inherited from the father)

my_mansion = mansion ("white")

So the constructor “def init(self, color):” will use the white color to build the mansion.

my_mansion = mansion ("white")
print (my_mansion.color)

Result = white

Now let´s call the other methods to check that we are using the inherited methods defined within mansion class

class mansion (house):
    def lights_on (self):
        self.electricity += 38
        
    def use_water (self):
        self.use_water += 19
    
    def ring_doorbell (self):
        print ("Ding-Dong!!")
        self.electricity += 3
        
my_mansion = mansion ("white")
print (my_mansion.color)#shows mansion color
print (my_mansion.electricity)#shows mansion electricity use
print (my_mansion.water)#shows mansion water use

my_mansion.ring_doorbell()#we call the methos ring_doorbell
print (my_mansion.electricity)

my_mansion.paint ("gold")#we call the paint method FROM the parent
print (my_mansion.color)

Result = 
red
0
0
rinnnnnnnnnggggggg!!!!!!
1
green


white
0
0
Ding-Dong!!
3
gold

Intro to Web Scrapping

Web scraping, is the process of retrieving or “scraping” data from a website. automatically, not manually. Web scraping uses intelligent automation to retrieve millions or even billions of data points from the internet’s websites..

If there is no API to download data from a site, I can use web scrapping

How does a website work?

To perform web scrapping, we need to know exactly how a website works, so take a look at this awesome intro to HTML here

We need to understand some TAGS and its structure, meaning, who is the parent tag and its children.

Sol et´s say we have <div> </div> , so this parent tag will have children…whre? Just inside them, so ANYTHING within <div> </div> will be children of that DIV tag.

<body>
    <div>
        <p>
            Mi message here
        </p>
    </div>
</body>

Now let´s take a look at this HTML

<body>
    <div>
        <p>
            Mi mesaage here
        </p>
    </div>
	<span>
			I am here
	</span>
</body>

Why are they called siblings? Because the TAGS <div> and <span> are at the same level, they are childs from <body>, but siblings to each other

In order to make easy to identify a TAG, we can put them something called ATTRIBUTES

These attributes can be Class or ID and are formed by a NAME and a VALUE

 <div class="main container">
        <p>
            Mi mesaage here
        </p>
    </div>

Now we can easily identify this <div> tag because it has a class named main container

To get a deep insight regarding HTML classes and ID attributes, please check this class tutorial and this ID one.

Client-Server Architecture

Take a look at this info to get some intro to it

URLs

When we type a site URL, like https://www.google.com/ it is an easy-to-read-URL, but if we do a search in Google we´ll get something like this

Why all that stuff? Because thr URL can be used to pass info regarding, in this example, our search.

Let´s analyze this

https -> protocol

www.google.com -> domain

/search -> Endpoint (identifies the action the server will perform, this time a SEARCH. We can concatenate several endpoints like search/users)

Then we have the parameters; this starts with “?”, so everything AFTER a ? are the parameters that will use the server to answer our request. Keep in mind that the parameters are a pair name-value separated by the “=” sign

We can have several parameters split by the “&” symbol

So in the example the variable “q”= web scrapping, sourceid = chrome and ie = UTF8.

All this info is received by the server and used to serveour request

Types of Web Scrapping and tools used to perform them

1 – Static scrapping (one-page) : when ALL info is in just one page and it does not load dynamic info.

Tools used: requests (to “ask” for data), Beautiful Soup to parse the XML and HTML we get, and Scrapy that gets done the two functions (request and parse)

: 2 – Static scrapping (several pages, same Domain) also called Horizontal scrolling (pagination )and Vertical scrolling. (product details).

Tools used: Scrapy

3 – Dynamic web scrapping: we´ll use some automation to fill data, to scroll and to wait for the page to load contents before scrapping what we need.

Tools used: Selenium

4 – APIs web scrapping:

Steps to web scrapping

1 – Define a Root or Seed URL, the main one from where to START the data extraction, maybe not the one to extract data, but the one from where we´ll start “travelling” to find the info

2 – Make a REQUEST to this URL

3 – Get the response from the previous Request (it will be HTML format)

4 – Parse the info to obtain what I am searching

5 – Repeat from step 2 with other URL within the same Domain.(may be obtained from the HTML response)

XPATH

To obtain the required info from the HTML response, we´ll need XPATH

XPATH is a language that allows us to buid expressions to extract info from XML or HTML data. I can search and extract exactly what I need from all the giberish we´ll get from our requests. We can search within the DOM elements in a number of ways.

Take a alook at this awesome tool to learn how to use XPATH

XPATHER

Now, we must understand how XML works, it is made of a structure of LEVELS, being these levels the nodes (HTML tags) and these nodes have sub-levels, or nested-levels called “child nodes”

Take a look at this piece of code; <body> is the ROOT level, and the Childs are: <h1>, <h1> , <div>, and <div>, but the 1st <div> tag has another child -> <p> and the 2nd <div> another child <span>

<body>
    <h1>Main title</h1>
    <h1>Another main title</h1>
    <div class="main container">
        <p>
            Mi mesaage here
        </p>
    </div>
	<span>
			I am here
	</span>
</body>

Now we can define our search axis to start a search, these axis are some parameters to filter the tags we are looking for.

If I use // (double slash) it will search within ALL levels of the document

If I do a single slash (/) it will only search within the root of the document

Note if found nothing, because <p> is a child of a child, it is not the root. This document has only one tag as root, and it is <body>

Ok, after defining the search prefix (//, / or ./) we must add the node we are searching, this is called a “step”. I can also define attributes to narrow the search even more.

This is done by adding [@ =] after the search prefix. Let´s say i want to find the <h1> tag with id title

Here an awesome intro to Xpath

TIP

We can run a “live” xpath request by opening web browser dev tools (usually F12 or right-click -> inspect)

, then go to “console” tab and run this code

$x("path expression")

Let´s see an example by requesting all <div> from root (//)

$x("//div")

Web Scrapping

Remember that in order to get data from a website we need TWO separate procedures;

1 – REQUEST the page/server

2 – PARSE the data we received

We´ll use some Python libraries to do this.

To extract info from one-static-page we´ll use 4 differente libraries:

Requests -> to obtain the HTML

LXML and beautifulsoup4 to parse the received info

Scrapy to perform the two operations; request and parse

to install a library just open a CMD -> command promp in Windows or a terminal if Linux and run

pip install library-name

or pip3 install library-name

or sudo pip install library-name (Linux)

or pip install library-name –user (windows)

We´ll also install (for dynamic sites)

Selenium

Pillow (to extract images)

Pymongo (to store data in DB)

In case you need Twisted to make scrapy work with windows, use this link

Requests full doc

LXML full doc

Pip full doc

Resources to keep learning

Data Structures AWESOME Cheat Sheet here

Github Repositories to Learn Python

For loop

While

Lists

Strings

Tuple-> immutable Lists; once defined we can not change it.

TIP: Tuples are Faster than Lists

SETS

Dictionaries

Functions

Archives

Adding info to a file

Creating a new archive (w)

Error Handling

Classes and Objects

Intro to Web Scrapping

How does a website work?

Client-Server Architecture

URLs

Types of Web Scrapping and tools used to perform them

Steps to web scrapping

XPATH

TIP

Web Scrapping

Resources to keep learning

Leave a Reply Cancel reply