REST APIs

• How to (and how not to) design REST APIs
• How to Create a RESTful API with Python and FastAPI
• How to Create Microservices with FastAPI
• system-design-primer
• Microservices-in-python-using-flask

• microservices showcasing Kubernetes, Istio, and gRPC.

Code Review Checklist

The rules are:

𝟭. 𝗔𝘃𝗼𝗶𝗱 𝗖𝗼𝗺𝗽𝗹𝗲𝘅 𝗙𝗹𝗼𝘄: Steer clear of tricky control structures; stick to simple loops and conditionals.

𝟮. 𝗕𝗼𝘂𝗻𝗱 𝗟𝗼𝗼𝗽𝘀: Ensure loops have a clear exit point to prevent endless looping.

𝟯. 𝗔𝘃𝗼𝗶𝗱 𝗛𝗲𝗮𝗽 𝗔𝗹𝗹𝗼𝗰𝗮𝘁𝗶𝗼𝗻: Favor stack or static memory allocation to dodge memory leaks.

𝟰. 𝗨𝘀𝗲 𝗦𝗵𝗼𝗿𝘁 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀: Keep functions concise, handling a single task. This goes along well with Clean Code practices (Signe responsibility principle).

𝟱. 𝗥𝘂𝗻𝘁𝗶𝗺𝗲 𝗔𝘀𝘀𝗲𝗿𝘁𝗶𝗼𝗻𝘀: Utilize assertions to catch unexpected conditions.

𝟲. 𝗟𝗶𝗺𝗶𝘁𝗲𝗱 𝗗𝗮𝘁𝗮 𝗦𝗰𝗼𝗽𝗲: Keep the scope narrow to maintain clarity. Use the smallest scope for your variables (e.g., private or protected in C#).

𝟳. 𝗖𝗵𝗲𝗰𝗸 𝗥𝗲𝘁𝘂𝗿𝗻 𝗩𝗮𝗹𝘂𝗲𝘀: Always check the return values of functions, handling any errors.

𝟴. 𝗦𝗽𝗮𝗿𝘀𝗲 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗼𝗿 𝗨𝘀𝗲: Minimize preprocessor directives for readability.

𝟵. 𝗟𝗶𝗺𝗶𝘁 𝗣𝗼𝗶𝗻𝘁𝗲𝗿 𝗨𝘀𝗲: Simplify pointer use and avoid function pointers for clearer code.

𝟭𝟬. 𝗖𝗼𝗺𝗽𝗶𝗹𝗲 𝗪𝗶𝘁𝗵 𝗔𝗹𝗹 𝗪𝗮𝗿𝗻𝗶𝗻𝗴𝘀 𝗘𝗻𝗮𝗯𝗹𝗲𝗱: Address all compiler warnings to catch potential issues early. This is often neglected in many projects!

Coding Guidelines

How I spend my time as a developer:

• 10% writing code
• 20% refactoring
• 70% reading code

This is why I optimize my code for readability while still in the writing stage.

It always pays off in the long run.

And I know the next engineer reading that code will be thankful.

So think about this next time you're hurrying to finish a task.

Here's a checklist in no particular order:

• Variables defined close to where they are used
• Fluent syntax structured vertically
• Names are descriptive
• 80 characters per line
• Early return principle

Keep Exploring!!!

Code Review pointers

• Purpose of the code, the approach taken, and any specific areas 
• Error and Exception Handling: Look for proper error and exception handling for robustness. The application should be able to survive erroneous situations.
• Separating source code, tests, and configuration files neatly.
• Separate Data Preprocessing, Modeling, and Evaluation Code: It’s crucial to separate stages of the development process into distinct steps. This allows both modularity and an easier debugging experience, as issues can be located more swiftly in well-defined sections.
• Design a Configurable Pipeline: A pipeline that can be configured allows for easy adjustments to be made and allows various models, data, and preprocessing steps to be swapped out with one another efficiently.
• Breaking up a configuration into separate classes can increase modularity. Each config class can be responsible for one part of the application's configuration. This makes the code easier to read and maintain.
• How easy to test, How standalone it is
• Key approaches applied / Do's and do
• Is it at a level people can come/contribute / Open to DS forecast tracking people
• Basics needed to work on it
• Basics needed to understand it / Any references/patterns to cross-check
• Does it have good enough documentation/testing 
• Have we implemented for one DB, or have we implemented for others to follow the pattern?
• Have we tested end-to-end in one flow?
• How do we manage configurations across data, ML, jobs, and results? Are these separate classes? 
• This current work is fetching results, running pipeline, fetch status, Is there equivalent work done?
• What flow do we need porting type work? Mimic the same patterns?
• How to test standalone?
• What minimal knowledge to operate on this

Keep Exploring!!!

EC2 - AWS Flask Deployment Example

Deploying Sample API code 

Creating an EC2 Free t2 instance

Basic Ubuntu

Allow Access

Save Public IP

Launch Instance

Run Updated Packages and the below commands

Ref - Simplified code example Link

Allow Access to Port 80. HTTP

Run from browser and access APIs

Keep Exploring!!!

Docker Custom Examples

Some minor fixes working with git project

Keep Exploring!!!

Best practice for container-based deployment system 

• Use a container-orchestration system such as Kubernetes or Docker Swarm to manage and deploy your containers.
• Use a container registry such as Docker Hub or Quay to store and manage your container images.
• Use a continuous integration system such as Jenkins or Travis CI to automate the build and deployment of your containers.
• Monitor your containers and applications using tools such as Prometheus or Grafana to ensure they are running optimally.
• Use a service mesh such as Istio or Linkerd to manage the communication between your services.
• Use a logging and monitoring system such as ELK or Splunk to track the performance of your containers and applications.
• Use a security scanning tool such as Twistlock or Aqua Security to ensure your containers are secure.
• Use a configuration management system such as Ansible or Chef to manage the configuration of your containers.
• Use a deployment automation tool such as Helm or Terraform to automate the deployment of your containers.
• Use a cloud provider such as Amazon Web Services or Google Cloud Platform to host your containers.

Keep Exploring!!!

Flask - Image Attachment - Samples - API

 

More Read - Deploying a Flask app on AWS Lambda with Zappa

Keep Exploring!!!!

Debugging / Code Practical Views

• Modularize and integrate it easier to debug
• Clarity of code changes vs perception of change, Know every key line of change, Why and how it works
• Easier to plug and test smaller portions of code as needed
• Documentation at method level on input/output
• Comment key lines of function in code (the core of logic), Keep versions of old key lines commented as needed. 80% of key logic may be in 2 lines of a function :)
• Logs to capture
• Spot the block that errors / Enable / disable the key lines / retain prior changes to spot low-level issues

Knowing purpose makes it easier to spot the line!!!

Keep learning code through a product lens!!

Learning from code reviews

An interesting read from link

My favorite list

Perspective #1 - You don’t need hundreds of engineers to build a great product

[Siva] - Have a set of ideas, experiment, fail, learn, unlearn, relearn, and Build a vision of the product not under the pressure of timelines

Perspective #2 - Simple Outperformed Smart

[Siva] - Start to crawl before you learn to run

Perspective #3 - Our highest impact findings would always come within the first and last few hours of the audit.

[Siva] - Functionality, Scalability, Performance matters

Perspective #4 - Business logic flaws were rare, but when we found one they tended to be epically bad

[Siva] - Product is a for customer need not for experimenting technology. Build what is needed for the customer, provide the customer experience

Perspective #5 - Quick turnarounds on fixing vulnerabilities are usually correlated with general engineering operational excellence.

[Siva] - Quality ideas come from the domain, data, and functional understanding. Think from the long term no near term fixes

Good Read - Link

• How does the number of reported defects in source code files correlate to source code quality?
• How much longer development time is needed to resolve an issue in files with low-quality source code?
• To what extent is the code quality of a file related to the predictability of resolving issues on time?

Keep Thinking!!!

Python coding review tools - Docker, Kubernetes

Good read on code quality Tools - Link

Picked and checked on below tools for example project

• Bandit - security issues
• Pylint - coding standard 
• pycodestyle - code style 

Download Project - Link

Result #1

Result #2

Result #3

This CI / CD 8th Lab video  was motivational to explore these tools. Need to re-read this session again.

Tool Links

pylint

bandit

pycodestyle

code - link

Todo List

Good Reads

• Exploring Shap - Link1, Link2
• Python Best Practices for a New Project in 2021 - Link1
• ML-Ops_ModelDeployment_k8s - Link1

Keep Exploring!!!

Examples Flask vs Fast API

Recently FastAPI I could see more posts/recommendations compared to flask API in a performance context. A basic example of implementation with flask vs Fast API. The format  /syntax, request differences you can spot by comparing them.

CI / CD

DevOps with Github

Makefile

Happy Learning!!!

Day #103 - Hosting a flask Rest API

This post is on hosting a flask / rest API. This requires flask package. In Python 3.4 It was straight forward implementation

To execute run the code in python console - python example_gist.py

The following links were useful
link1, link2, chromeextension

REST API Interview Questions (Beginner Level)

REST API Interview Questions (Advanced Level)

Happy Learning!!!

Weekend Reads - API Testing (Web Service Testing, Inspect Http request, Rest API Testing) - Free tools

Very good presentation on compiled list of free tools for API Testing. More Details - Free API debugging and testing tools you should know about

Tools List from the presentation and SO reads

• Rest-assured - Java DSL for easy testing of REST services
• Retrofit - A type-safe REST client for Android and Java
• RequestBin - Inspect HTTP Requests
• Hurl.it — Make HTTP Requests
• ngrok captures all traffic - Raw request/response bytes
• mitmproxy: a man-in-the-middle proxy - Intercept, modify, replay and save HTTP/S traffic
• Postman is a powerful HTTP client to help test web services easily and efficiently
• Mock your HTTP responses to test your REST API
• Chrome Rest Console - REST Console is an HTTP Request Visualizer 
• SO - Online testing a REST service
• Simple REST and HTTP API Client for .NET

While reading through the list again went back to check on SOAP, Rest Basics. API testing / Web Service testing we will be looking into only aspects Rest, SOAP based web services. Summary based on StackOverflow readings, posts. References - StackOverflow reference answers link. More details Pls check reference link. (Consolidated Answer and Detailed short summary listed below)

Rest	SOAP
REST is over HTTP. REST has no WSDL interface definition	SOAP can be over any transportprotocols such HTTP, FTP, STMP, JMS etc.
REST stands for Representational State Transfer. REST approach uses the standard GET, PUT, POST, and DELETE verbs	Simple Object Access Protocol (SOAP)  SOAP builds an XML protocol on top of HTTP  / TCP/IP.
REST is good for getting a blob of data that you don't have to work with	SOAP describes functions, and types of data. If you want to get an object, SOAP is way quicker and easier to implement
Typically uses normal HTTP methods instead of a big XML format describing everything	Has several protocols and technologies relating to it: WSDL, XSDs, SOAP, WS-Addressing
REST plays well with AJAX'y web pages. If you keep your requests simple, you can make service calls directly from your JavaScript, and that comes in very handy.	SOAP is useful from a tooling perspective because the WSDL is so easily consumed by tools. So, you can get Web Service clients generated for you in your favourite language.

More Reads

HTTP vs REST vs SOAP

REST and SOAP: When Should I Use Each (or Both)?

From AWS Blog - 80% REST / 20% SOAP usage pattern

Example - Learn REST: A Tutorial
Restful Load Testing

Happy Learning!!!