Python is quite dynamic. Variables can reference strings, then lists, then sets... It's an intrinsic characteristic of the language.
I was there, thinking that it could be possible. Never found a safe trail for achieving this, and for a long time, I thought it was just me. Some think that Type Hints could be a starting point of having such feature, without understanding the concept of Type Hints and annotations.
Some people might think that Data Classes could provide some solution, and nope: Data Classes are just a more elegant way of creating Classes, implementing __init__ and __repr__ methods by default, among other functionalities, but still, it does not take care of type enforcement at runtime level.
The best that you could have is type enforcement at instantiation level, being a Data Class or not, but still, you can pass the expected data while creating the object, later on changing the instance variable, without any validation penalty:
Pydantic does this kind of type enforcement gracefully at object creation, not runtime:
It's unlikely to happen in the near future, to be honest. And if it happens, it would be a great change for the specification of the language. Data type enforcement is a great advantage in compiled languages (not our case here), then accidents of defining an integer to a char type variable, not being allowed by the compiler.
To keep a variable consistent in regards with its data type, is something we must do by default while developing in Python. IDEs and linters can help on that. And that's the best that you can have.
I shared something on this thread a couple of weeks ago:
If type enforcement at runtime level is something you really need on your project, you need a different programming language, considering the trade-offs of not using Python.
It would be nice to have such thing in Python, one day. Hope that it will not remain as a wild dream that we all kind of have.
Sometimes it's on our side, specially when we are young, yet sometimes it could be against our will. But it's unbeatable, unstoppable, for poor, rich, old and young. Its perpetual existence in our lives, almost gives and idea of a living entity, isn't it? But beyond our lyrical view, it's more practical than we generally imagine.
For us, Developers, from tasks that we need to finish of our sprints, to date and time computations that provide information from applications, time plays a big role.
I was truly wondering on some aspects about the perception that we can have from time, from science to software development, and this article came out. It could be an amalgamation of multiple subtopics, but it would be unfair to discuss about time computations in Python or any programming language, without writting about other time related things that precede the invention of the digital computer.
Earth's Spins
A complete rotation of Earth on its axis, it's a day (not exactly 24hs), divided in hours, minutes, seconds, etc., and Earth revolves in orbit around the sun, resulting in a Tropical or Solar Year. Even before knowing this astronomical perspective, Mexicas, Egyptians, Greeks and many other civilizations, were counting days, being by the cycle of Sun, Moon, or even by other more simpler events, creating their own devices for timekeeping.
Counting with Precision
In fact, Earth's revolution around the Sun has extra hours and minutes, which are later on added as the 29th day of February of a Leap Year. Worth to observe that Earth's rotation speed varies in response to climatic and geological events, resulting in Leap Seconds, which are gradually added to the Coordinated Universal Times (UTC), where Atomic Clocks have an important role on adjusting the time as we know, providing precise measure of a Second, based on the transitions of the Caesium-133 atoms.
Analogical clocks are useful, yes, even though they get out of sync. Before the Atomic Clocks, they were there, with great engineering on their manufacturing, like this UTC Clock designed and created by the great Mexican engineer, Alejandro Olvera Charolet, using cogs, pendulums, relying upon the force of gravity.
UTC Clock by Alejandro Olvera Charolet at Centanario Clocks factory (Zacatlán, Puebla - Mexico)
But since when should we count the days?
Besides the Gregorian Calendar, there are hundreds of calendar systems: Iranian, Japanese, Korean (South), Juche (North Korea), Tibetan, etc. What all of them have in common is: an epoch. A starting pointing for counting the time, generally being an event, an epoch.
On the Gregorian Calendar, the birth of Jesus Christ, is the epoch where the time starts to be counted, having the notion of time before (Before Christ) and after this event (Anno Domini). The fall of the Persian Empire was in 330 B.C., and the Colosseum was built in Rome between 70 ~ 80 AD: two events, one before and other after the birth of Jesus Christ.
That being said, computers should know from which epoch the time starts to be counted too for a matter of computations related with date and time. Operating Systems have different dates as the epoch. On Unix and GNU/Linux like OSes, January 1, 1970 00:00:00 is the epoch. The engineers Dennis Ritchie and Ken Thompson between the 60's and start of the 70's, where discussing the definition of an epoch for Unix and the New Year's Day was the option, later on inherited by all flavors of Unix 'n GNU/Linux created so until now.
Unfortunately, we don't have so much time until the Year 2038 problem, at least for the legacy 32-bits architectures, for the maximum signed integer supported by 32-bits computers, will reach the timestamp 03:14:07 on Tuesday, 19 January 2038. On the next second, 32-bits computers will start counting the time at 20:45:52 on Friday, 13 December 1901.
Software Development, Time and Scenarios
It's a challenge for us too. We have to deal with it, in terms of hard and soft skills, developing software that transforms and calculates time spent, while keeping an eye on the sprint, for time is running, and so the sprint. We have also Time Complexity serving to estimate the amount of computer time an instruction may consume, therefore understanding the worst case scenario of our code, helping us to optmize our code to the most efficient instructions, having some relation with Software Profiling (subject for another post).
Some tasks that we face, involving time computations:
data ingestion, normalizing and storing data and timestamps in databases
processing of programs/web services log files, measuring intervals between events
exposure of database records and their timestamps through APIs
And by the way, hardware clocks/computers must be synchronized with Time Servers connected with Atomic Clocks via a network protocol called NTP. As long as we have NTP protocol working fine (no firewall filters) and Time Servers synced with Atomic Clocks, hardware clocks will always be synced, so as the date and time of our applications, their events, everything precise, from the moment that you buy a product, to the moment that you received the package in your house.
Time computations with Strings? Nope.
String representations of time are hard to be calculated. Converting number representations to integer data types, then performing calculations, nope, that doesn't work. Fortunately, standard libraries from many programming languages have their own implementations for converting string representations of date, time or timestamps, to objects, so we can access years, days, minutes, seconds, etc., as attributes, making it way more easier to access and compute these values.
While interacting with external data sources (REST APIs, .csv files, etc.), things start to get complicated, for we may encounter a variety of date/time presentations (some not following ISO 8601 standard), and the evaluation of how the timestamps are presented is mandatory, in order to convert these representations into datetime objects, these being informed about the timezone (AWARE) or not (NAIVE), doing the required computations and storing it on a database system along with all data processed.
Human Readable Timestamps
The opposite operation. While retrieving database records and performing the necessary computations, datetime objects must be converted to human readable strings. On Python, it could be using strftime() or isoformat(), it depends on the business requirements.
Possible Scenarios While Creating/Consuming APIs
Not so common, but an API with a single endpoint, pretty much representing the flow between User <-> Webservice, serving data and timestamps that participate on relevant computations, delivering more data to the user or to the Frontend of the application (User Interface):
On the other hand, here's a process that performs a request to the API, doing an extra computation, providing an information that wasn't exposed by the API's endpoint (observe the intentational computation between AWARE and NAIVE datetime objects, for a matter of demonstration):
Final Thoughts
It's funny to think about how many things are related with time, specially for us, Developers. Even though we know how to make time computations, extracting information and insights from events, we are just touching the surface of what time really means, beyond our professions.
We may ramble on about the philosophical and technical aspects that this 'entity' constantly plays in our lives, and we may find answers, methods and tricks to make our lives easier while dealing with this titan, but it's the best that we can do so far, in order to invest our efforts on things that most matter for us, for we will not live forever, and neither our loved ones.
I heard once that, if we could extend our existences, then we could beat the time (hahaha). Just making our Telomeres more resistant to the natural shortening, so it will delay our natural end, it's not immortality. We want to live longer, but why?
"It is the knowledge that I am going to die that creates the focus that I bring to being alive. The urgency of accomplishment. The need to express love — now, not later. If we live forever, why even get out of bed in the morning? Because we always have tomorrow. That's not the type of life I want to lead."
Therefore, this Grim Reaper, plays an important role on how do we understand our existence.
The existence of humanity expressed through counted astronomical events, understanding for how long things existed (or still exist), or in which period of our calenders an event has happened. There's no deity or entity here: it's just us, counting events while we get old. And if that bothers you, think for a moment and realize, that you still have time :).
One may find her/himself creating scripts with initial and final instances of datetime.now(), executing a block of code in between them and then obtaining a delta between the two timestamps, in order to understand how long the code will take for its execution, or measuring performance between two or more approaches. I remember doing this in my early days of programming. It's not a bad approach: it actually works pretty well, but Python is a "batteries-included" language, with an extensive set of libraries available on its installation, like timeit, that will do the job for plenty of tasks that we come across.
Timeit does an excellent job, indeed. It executes a block of code N times and it repeats this loop R times as well, providing the best execution time while, for example, running a piece of code 100000 times, in 5 separated rounds of execution.
Running via Command Line
Here are some quick examples of how to use via command line.
Default values
Defining 10 rounds
Defining 10 rounds of 100000 executions
With this understanding, for example, we can compare the different methods of joining strings, and how efficient each one them can be:
Testing Module Functions and Comparing Approaches
Here's a more formal approach, testing a module and a couple of functions from it. I'm now comparing 3 different ways of joining strings, being it via concatenation or interpolation:
Seems that the first approach is more efficient, indeed. To my surprise, I've always thought that ''.join() would be more efficient than normal string concatenation. Worth to notice that each execution of timeit.repeat(), returns a list, that we can use as series for plotting on a chart, so we can have a more pictorial presentation of time differences between each approach.
MORE Comparisons...
Lists have different ways of being extended, incremented, etc. Let's review this too.
There isn't so much difference but, it's fair to say that extend() isn't the fastest method of extending a list. For better spotting the difference (which is pretty small in nanoseconds), let's plot these series on a chart.
Plotting
Just a couple of general settings for a Matplotlib chart, easily configured by looking at Matplotlib official documentation:
Here the results:
Indeed, incrementing is the most efficient method for extending a list.
But what about the joining of strings through concatenation or interpolation?
Here's an adaptation of the previous code, but using the string functions observed before:
And the difference in nanoseconds scale, is even bigger:
Any time difference represents a big difference, when we think about millions of executions of a instruction, even for the list extension methods. Using the most efficient method in order to achieve something, it's a practice to be maintained and it always pays off. But if the code isn't repeated so many times, it's ok to use whichever method you like, as long as the code is clean and objective (and documented, please),
Final Words
After using timeit, it's hard to think about other methods for measuring the time efficiency of blocks of code, or even small pieces of code. It's not intended to be a full profiler tool, but at least, gives you a clear perspective of how fast a approach or method can be, spotting minimal differences between similar approaches, and these small differences, will certainly represent more time consumption to your program, if the code is exposed to thousands of iterations.
Over the years, developing software, I sometimes wonder how much do I know about OOP. There's so much components and applications that one builds, that are just functions or a coupĺe of classes from a regular library import. I guess I'll start some review here. Maybe this could serve to anyone, maybe not.
Chairs as Objects
Just like a chair can be assembled while reading the instruction manual, an Object can be "constructed" by a compiler, based on the definitions of a Class. You can think of a chair being an Object and the instruction manual being the Class that orients the individual on assembling it.
$ cat oop.py
class Chair:
def __str__(self):
return "A chair was assembled."
chair = Chair()
print(chair)
$ python3 oop.py
A chair was assembled.
Any object or thing, has its own properties, or characteristics, just like a chair have a color.
$ cat oop.py
class Chair:
def __init__(self, color):
self.color = color
def __str__(self):
return f"A {self.color} colored chair was assembled."
chair = Chair(color="black")
print(chair)
$ python3 oop.py
A black colored chair was assembled.
Actions, Features, Properties
Objects can have functionalities or actions, and so Chairs can have, like a lever to lower and raise the seat, or cannot. Therefore, we can assume that a regular chair is different from an office chair, which can rotate or lever the seat. That means, an office chair has a particular set of actions, that a regular chair doesn't have.
$ cat oop.py
class Chair:
def __init__(self, color):
self.color = color
def __str__(self):
return f"A {self.color} colored chair was assembled."
class OfficeChair:
def __init__(self, color):
self.color = color
def lever_down(self):
return "Office chair seat goes down."
def lever_up(self):
return "Office chair seat goes up."
def rotate_left(self):
return "Office chair rotates to the left."
def rotate_right(self):
return "Office chair rotates to the right."
def __str__(self):
return f"A {self.color} colored chair was assembled."
chair = Chair(color="black")
print(chair)
office_chair = OfficeChair(color="red")
print(office_chair)
print(office_chair.lever_down())
print(office_chair.rotate_right())
$ python3 oop.py
A black colored chair was assembled.
A red colored chair was assembled.
Office chair seat goes down.
Office chair rotates to the right.
Code Reusability and Organization: The Power of OOP and Inheritance
Although being different types of chairs, both share a common attribute: color.
An office chair is a chair after all, so it's fair to say that an office chair is a derivation of a regular chair: it inherits characteristics and has the same purpose of regular chair.
Therefore, an inheritance can be established from Chair to OfficeChair, eliminating the need of defining a specific color attribute to OfficeChair or even defining a specific __str__ method.
For color attribute, with a little bit of polymorphism, we can redefine the implementation of __init__ method inherited from Chair class, calling __init__() from Chair via super() method, setting the color attribute of Chair, without need of setting specific color for OfficeChair.
Observation: it can also be said that OfficeChair, inherits attributes and methods from Chair.
$ cat oop.py
class Chair:
def __init__(self, color):
self.color = color
def __str__(self):
return f"A {self.color} colored chair was assembled."
class OfficeChair(Chair):
def __init__(self, color):
# __init__ was already inherited by Chair and here, it is modified.
# Instead of setting a class attribute of color for OfficeChair, it just
# sets the attribute of the upper class (Chair)
super().__init__(color)
def lever_down(self):
return "Office chair seat goes down."
def lever_up(self):
return "Office chair seat goes up."
def rotate_left(self):
return "Office chair rotates to the left."
def rotate_right(self):
return "Office chair rotates to the right."
# This method is unnecessary, for OfficeChair inherits __str__ method from Chair class.
#
# def __str__(self):
# return f"A {self.color} colored chair was assembled."
chair = Chair(color="black")
print(chair)
office_chair = OfficeChair(color="red")
print(office_chair)
print(office_chair.lever_down())
print(office_chair.rotate_right())
print("The color of the office chair is: ", office_chair.color)
$ python3 oop.py
A black colored chair was assembled.
A red colored chair was assembled.
Office chair seat goes down.
Office chair rotates to the right.
The color of the office chair is: red
Grandfather, Father, Son: more inheritance...
Thinking on office chairs, there are ergonomic office chairs, that have a couple more of movements and adjustments. They can lean back or forward, and even have arms that go up and down. But after all, they have also the same functionalities or actions as a regular office chair.
We can now think about three types of chairs:
regular chair (Chair)
office chair (OfficeChair)
ergonomic office chair (ErgonomicOfficeChair)
They are all chairs, so the inheritance easily applies from Chair to ErgonomicOfficeChair, which has its own set of actions like lean back or lean forward, but it also rotates or the seat can go up and down. In other words, an ErgonomicOfficeChair has the same functions as an OfficeChair, and through inheritance, we can use the inherited methods, instead of writting them again.
Also, from ErgonomicOfficeChair, we set the color attribute from Chair and we can use it, for the inheritance of Chair to OfficeChair, is accessible via ErgonomicOfficeChair, which inherits from OfficeChair.
$ cat oop.py
class Chair:
def __init__(self, color):
self.color = color
def __str__(self):
return f"[INFO]: A {self.color} colored chair was assembled."
class OfficeChair(Chair):
def __init__(self, color):
super().__init__(color)
def lever_down(self):
return "Office chair seat goes down."
def lever_up(self):
return "Office chair seat goes up."
def rotate_left(self):
return "Office chair rotates to the left."
def rotate_right(self):
return "Office chair rotates to the right."
# This method is unnecessary, for OfficeChair inherits __str__ method from Chair class.
#
# def __str__(self):
# return f"A {self.color} colored chair was assembled."
class ErgonomicOfficeChair(OfficeChair):
def __init__(self, color):
super().__init__(color)
def lean_back(self):
return "Office chair leans back."
def lean_forward(self):
return "Office chair leans forward."
chair = Chair(color="black")
print(chair)
office_chair = OfficeChair(color="red")
print(office_chair)
print(office_chair.lever_down())
print(office_chair.rotate_right())
print("The color of the office chair is: ", office_chair.color)
ergo_office_chair = ErgonomicOfficeChair(color="blue")
print(ergo_office_chair)
print(ergo_office_chair.lean_back())
print(ergo_office_chair.lean_forward())
print(ergo_office_chair.lever_down())
$ python3 oop.py
[INFO]: A black colored chair was assembled.
[INFO]: A red colored chair was assembled.
Office chair seat goes down.
Office chair rotates to the right.
The color of the office chair is: red
[INFO]: A blue colored chair was assembled.
Office chair leans back.
Office chair leans forward.
Office chair seat goes down.
All this review deserves more writting, for a next article. For now, this pretty much gives an idea of the power of code reusability and organization that OOP provides.
Even though Python is not fully object-oriented due to lack of strong encapsulation, isn't a week language in terms of OOP. That will be something more for another article. See ya.
You might already heard about CVE or Common Vulnerabilities or Exposures, which is an initiative sponsored by DHS (Department of Homeland and Security) and CISA (Cybersecurity and Infrastructure Security Agency), overseen by MITRE Corporation . All CVEs are listed on this website maintained by MITRE. If you want to know more about CVEs, how they are discovered, registered and how they are evaluated and approved, this document from RedHat has one or two words about it. Also, PYPA maintains the advisory-db project, for searching vulnerabilities of Python packages (this is where pip-audit comes into play).
Security is a limitless topic
It's not just about having an armored IPTABLES/Netfilter, and Nginx well configured with HSTS and other secure headers, and tunned Linux Kernel, and your Backend application demanding authenticated requests of 99,9% of your endpoints: from the packages that you download through APT, to PyPi packages that belong to your Python project, one of them could have a vulnerability, or a CVE registered and document.
In terms of security, we can go forever. It's a topic always on the table, from Infrastructure to Software, and it will always be like that.
Before pip-audit
You might be just like me, reviewing package by package from your requirements.txt with 50 different packages on it, searching on CVE listing websites like the one maintained by MITRE or even this one. What a pain, right? To review all packages (one by one) and document the vulnerabilities (if any) on your JIRA card, but, there's no other way. True that some services are available for searching PYSECs, but still not that pragmatic, in my opinion. I always felt more confident on searching CVE lists, manually. If you protect your API endpoints with all that you can, you can't avoid the audit of your project packages. You just can't..
After pip-audit
I think that everyone who cares about security, was craving for something like this. It was announced by Dustin Ingram yesterday, the stable release of pip-audit.
From time to time, I'm involved in security audits on companies that I work (I was a SysAdmin before being a Software Developer), so I can guarantee, that this comes from heaven:
Final Words
I'm pretty sure that this tool, will become a standard for all of us who develop software or maintain Python projects (open source or not). It's true that there's a great dependency on security at infrastructure level, being from network traffic to webservers and Operating Systems, but it's common to observe Software Developers not being concerned on reviewing security of packages, even though they are really worried about how protected is the API from a variety of vectors.
Your platform is also secure, by having components without vulnerabilities (keep in mind). If you don't audit your packages, it's time to make a change, now being more efficient. Cheers.
It's always a good time, to open up a terminal and expressing calculations through code, just as any scientist would do, with the exception that I'm no scientist (at all), although I kind love Computer Science. I'm Software Developer so, it is expected, I guess.
Being from another country, speaking another language and having a different education at school when I was young, it feels a little bit different to read and identify mathematics content like Greatest Common Factor, Factors, and so on. Reading this article, I throught that would be nice to make some code that finds the factors of a number, and even better, the GCF from a list of numbers. I'm still wondering on what would be the best code, in terms of efficiency and with less time complexity, though.
Factors
Well, I started to perform a procedural code, 2 or 3 lines, but remembering how powerful are List Comprehensions (good source btw), many iterations and data strcutures can be performed with a single line.
Mission accomplished, with an easy and elegant code:
GCF (Greatest Common Factor) - 1° Round
This takes me back to my childhood, when I was 12, I guess. Easy to put on a paper. To express the solution with code? Somewhat, at the beginning. Here's the 1st approach, relying on the return of get_factors() function (list of factors).
Mission accomplished, but not meaningful (why using a list of lists of factors ?):
GCF (Greatest Common Factor) - 2° Round
No need of dealing with lists. It's just about numbers, just as you do on a paper. A couple of variables were changed for the sake of readability, and the rest stills the same. Mission accomplished and much better, isn't it?
Was an interesting task. It's a mix of childhood memories of mathematics from high school with my perspective nowadays as Software Developer, scratching the surface of Computer Science, but passionate on finding answers through programming languages, which in my case, is Python, most of the time.
Generally, we all iterate through a variety of objects in Python, like:
Tuples
Lists
Dictionaries
Strings
Sets
But besides theses objects, you can create your own iterable object! Cool huh?
Not that often you might run on this situation, but it's good to know how to build your own iterator and also, you will learn a bit more of Python internals and how the iteration works behind the curtains.
The __iter__ and __next__ methods
Data structure objects like the ones listed above, are all iterable objects. You can get an iterator for any of them, by using the iter() method, and then iterating over it with next() method. If all elements from the iterator were called, then a StopIteration exception will be thrown.
Your Iterator
Built as class, your iterator should have __iter__() and __next__() methods implemented: one for initializing your iterator object, and the other for providing the current iterator value, also calculating the next iteration:
The problem here is that, without a condition, this iteration can go forever:
Depending on your code, you might want to have a condition externally expressed, but most of the cases, the condition of how many interations will be supported by the iterator, are defined on the class which provides the iterator.
Max Number of Iterations
Just as any class, you can define the __init__() method for the iterator class, where you can define the limit of the iterator:
Final Words
Hope you had fun while reviewing this topic and hope that it might help you some day. I decided to write it here, for I went thtough some situation where implementing an iterator was necessary, and here's a record of something that I initially tried, in order to understand how to build one.
From now on, everytime that you iterate through an iterable object, you can have an idea of what's going on with this object, how the data is being processed, stored, and understand that, there might be very specific scenarios where you would like to implement your own iterator.
For more examples and resources, here's a cool document from Python official documentation.
On my first steps as Software Developer, I used to develop my own implementations of JSON validation, which worked great, but some JSONs were complex to validate, increasing complexity on improving the custom validators, becaming difficult to keep up these implementations. After some research, I discovered json-schema, which is a specification with drafts written for IETF and has implementations for many programing languages.
Here, I brought some examples on how to validate JSON data against schemas, based on its implementation for Python, including a small Web Application written in Flask.
JSON Schema defines the media type application/schema+json for
describing the structure of other JSON documents. JSON Schema is
JSON-based and includes facilities for describing the structure of
JSON documents in terms of allowable values, descriptions, and
interpreting relations with other resources.
Simple Validation => validate() method
This method throws out an exception, if any violation was detected, between data and schema:
Simple Validation => is_valid() method
I'd rather use this method instead of the prior, since that it just determines if a JSON is valid:
Lazy Validation => iter_errors() method
Probably one of the most interesting validation mechanisms, where you can gradually iterate over possible errors, without causing exceptions, and no need for try/except blocks.
The JSON schema specification has a series of versions, and here, we are using the Version 7:
Required Fields
Simple as it is, some fields are required, and if some of them are missed, the validation will fail:
Web Application Scenario
Here, a taste of JSON validation schema with JSON data over POST HTTP request:
The requests can be performed via Postman of just with simple cURL requests:
Command Line Scenario
You might want to validate some JSON files by hand, having a schema defined in a file as well:
Final Words
These are just a few examples and there are other functionalities like annotations, minimum and max length of fields, arrays validations, etc., that weren't exposed here. Although the examples here are simple, they have applicable features for complex structures as well, thanks to the solidity of JSON Schema standard.
If you have any question or suggestion, please, leave a comment. Thanks!
It was supposed to be something short, but it became an article, since that the topic could be quite extensive. It might be a tour for you, and I hope you might find something useful. It also has a tutorial with a lot of images and few text (don't worry). Here we go.
First Steps with Cloud Functions
I recently started to work with Google Cloud solutions, specially using Cloud Functions, for a US based client, developing some components for its COVID-19 assessment and notification system. My experience with Google Cloud solutions was equivalent to 0 before this project. I was quite used with AWS for the last 2 years (EC2, S3, RDS, Lambda, etc.), and I felt a little bit lost at the beginning and to be honest, unhappy. It was quite of a journey, to bring things fast on a platform which I just started to work with and the pressure for releasing the product, was something to add up on an almost traumatic process. Fortunately, the day to day experience made me able to get through it successfully.
As a side task, I decide to create my own Google Cloud Free Trial account, make some tests and understand a little bit more about Google Cloud, on a sandbox environment. While I still have some credits, I decided to bring some tips and instructions, which can help someone with creating its own Google Cloud Free Trial account and deploy a simple Python-based Cloud Function.
What if Google charges me all of sudden?
Unlikely to happen. Google Cloud Free Trial is provided with U$ 300 for 90 days of usage. Which ever comes first, spending all the credits or reaching 90 days of usage, you won't have charges on your credit/debit card. The access to Google Cloud will be blocked and only when you decide to unblock it, you'll be charged, depending on usage of the service and its pricing. While setting up you're first Cloud Function, you have to enable billing, but don't worry, since that the billing will only consume the credits from your Free Trial.
Can I make big things with Cloud Functions?
Although I'm presenting just the snippet from Cloud Function editor (and also a small modification of this), you can have key components of your platform running as a Cloud Function, without setting up a server and setting up a lot of stuff. Using this, you're starting to have Microservices/Serverless oriented architecture, in other words. In a real case scenario, you can have a Cloud Function which receives a POST or GET request (Query String) from your main web application, and with the right logic, your Cloud Function can use the information provided on the request to lookup for some specific data from your database system, and do some businness rule with this like sending an email with a report to a costumer or notifying your main web application about the end of service for a user which has an expired service. And for such scenarios, I recommend to configure authentication on your Cloud Function in order to safely access its URL.
You Have to Know About Flask
Flask is a microframework for developing web applications and web services based on Python. It's very simplistic and easy to manage, letting you start little by little, instead of all steps and application structure that Django demands. Both have its own bennefits, but Flask is the framework which will handle HTTP requests to your Python-based Cloud Function. Here's a taste of a minimal Flask application and the handling of HTTP methods. You might also want to understand a little bit about flask.request object, which handles the HTTP request performed to your Cloud Function, which is also the parameter of your entrypoint function.
So we don't need to have Servers anymore ?
Besides being Software Developer, I'm experienced on infrastructure services and can guarantee, that a server designed and configured by an experienced System Administrator, could be much more secure, reliable and faster than a Cloud Function, allowing you to debug, secure, identify and improve your application performance with a flexibility that a Cloud Function won't provide to you. But you need someone to deal with this and the costs might be different from a Cloud Function. To have or not to have Serverless architecture, demands a deep look on all pros and cons, instead of following the buzz of Serverless and Microservices that Google, AWS, etc. makes. Might be a good thing for companies which don't have one or two individuals dedicated to infrastructure or startups, but would it be the right thing to your business?
Requesting Free Trial
Access Google Cloud and click on "Get started for free":
After login, a screen similar to this one will be prompted:
Navigate to Google Cloud Functions option:
Enable billing option (CTRL + click, for opening another window):
Come back to the Google Cloud Function window, reload it and start to create a function:
Setup a Google Cloud Function
Configure the name of the Cloud Function and allow unanthenticated access for its URL:
For now, it won't be necessary to setup environment variables:
Select Python on the Runtime menu:
No need to define additional packages on requirements.txt. Click on DEPLOY button:
Wait for the green check of the DEPLOY and click on the created function:
Click on TRIGGER menu and click on the URL defined to your Cloud Function:
Or perform a request from your terminal with curl:
For simulating a POST request or a GET with a Query String, you can do something like this:
On the screen where the URL is provided, click on VIEW LOGS for observing the activity of the HTTP requests performed at the Cloud Function:
Environment Variables
Using Environment Variables is a power up to your code and also to your Cloud Function. You can provide dynamic changes on your code, like setting up a different database address and credentials, or turning on/off other functionalities from your Flask app. For your local environment, it would be better to have all the Environment Variables necessary to your app, defined on a .env file, loading it via Python Dotenv. Here's an example below, using the same code, but defining an Environment Variable and obtaining its content inside the Cloud Function, just as you might normally do on your local environment.
After clicking on your Cloud Function, follow this click path and define your Environment Variables: EDIT => VARIABLES, NETWORKING AND ADVANCED SETTINGS => ENVIRONMENT VARIABLES:
Modify your Cloud Function by importing os library, then obtain the content of your defined Environment Variables via os.getenv(). After editing, click DEPLOY again for updating the Cloud Funtion:
Performing a simple GET request, without a query string, will bring the environemt variable DEFAULT MESSAGE. For POST with application/json as Content-Type and GET with Query String "message", it will bring the message sent via the request:
Have a mockup before a Cloud Function
Being on dev, stage or production environment, it's a good thing to guarantee that everything is working fine locally, before moving on with your Cloud Function. Here's a Gist for helping out with mocking up your Cloud Function:
Final Words
Indeed, relatively easy to setup a Cloud Function and a handy solution in order to bring up your component alive, without dealing with all the setups on an instance/server. Serverless architectures are here to stay and spreading very fast. From this Python-based Cloud Function example, Thousands of products from startups to established businesses, are delivering great solution on a Microservices oriented architecture, and taking advantage of what it has to offer, but it's not the end of the classic Server architecture. Remember, that although a Serverless architecture is easy to deal, maintain and also to scale, it doesn't provide the power and reliability that you can achieve while tunning up manny aspects from your administered server, from Kernel parameters to specific configurations to your Web Server, bringing more control, security and reliability to your products, not having a blackbox which you don't have control over it.
As an advice: analyze carefully, pros and cons. Don't get excited about the buzz. Both approaches have great things to offer, but only through a deep analysis, you can determine which one is better to your business, or maybe both.
Questions, doubts or improvements, feel free to leave your comments. Thanks for reading.
A code that fails and no further information is provided, is a situation of despair: no Traceback, no nothing. This is what a Segmentation Fault provides.
I was almost starting to questioning my career choice, while I was in a hurry for delivering a code ASAP and this catastrophe happens. What a nightmare.
It happened while using MySQL connector for Python. The reason: a Bug on version 8.0.8 where the connector crashes, if you have imported Python random library or if any component of your project has it on its implementation. In my case, I was using Python Client for Google PubSub, which has Python random library on its implementation.
To identify this all this scenario, it was only possible with faulthandler library:
