Gradient Image Processing is built as a single dedicated service that is applied for image processing. The solution supports multiple services that perform the upload of original images and retrieve processed result images. The result images are retrieved from the Gradient Image Processing service. The intended application of this version (and current implementation) is web traffic optimization on web pages, where images are compressed and scaled to the appropriate size.
Top Level overview of Gradient Image Processing is presented in the following illustration:
Illustration is explained in more details in the following sections.
Under consumer services are considered services, software solutions, and technical implementations that utilize Gradient Image Processing. Every service that needs processing (and retrieval) of images is able to communicate that with Gradient Image Processing.
For example, some "CMS Service" might be using the Public API Layer to perform an upload of an image and get designated result images. Later on, it can download the resulting image when needed. After upload consumer service is informed about result images and necessary meta-information about them.
With all this in mind, every work with images that consumer service needs, should be performed inside the Gradient Image Processing Service.
For consumer service to manage the images, authorization is needed. The authorization layer recognizes three types of actors in the system: Admin, Client, and ClientApp. Consumer services are client applications (ClientApp).
Admins and Clients can use Dashboard (built-in GUI interface), which provides standard CRUD operations over Admins, Clients, and ClientApps. Every inserted Client Application gets automatically generated AppID and SecretID.
Client Apps, with credentials, are able to upload original images that need processing and get information about result images for later usage. Additionally, Client Apps can request deletion of the original image and all its resulting images.
When the original image is uploaded, processing starts. The image is ready for processing, and the result images are created. Gradient Image Processing opens one WebSocket server through which Admins, Clients, and Client Apps can track (listen to) the progress of processing.
Internal API currently supports Image Configurations management, scaling, and compression of images, with separation between original and runtime images. Runtime image is the resulting image of compression.
The design supports multiple data storage for Client Application keys and Image information and configurations. Original and runtime images have different characteristics, so Internal API is designed with the possibility of having different storage types.
Based on the top-level picture, expected actors in the system and operations needed to be performed by Gradient Image Processing, a detailed feature set is defined. That feature set is visually presented in the following illustration:
After defining the feature set, for achieving the goal of extendable, scalable, and maintainable image processing service, the following technical goals are defined:
By considering enlisted technical goals, final solution architecture is built and implementation is performed.
Decision has been made that the solution utilizes Typescript as a programming language. Considering the extremely modular approach that is performed, platforms and frameworks that are explicit about modules are picked. The criterion for the framework choice is defined by the amount of provided module abstractions in it. The selected framework must support a module description, which includes its internal services/components, its exported components, and its dependency modules. Frameworks that satisfy this criterion are Nest.js for the backend and Angular for the frontend implementation.
Related to the persistence layer, a relational database is picked and provided tooling about data integrity is fully applied. For Gradient Image Processing, PostgreSQL is utilized.
When it comes to processing that needs to be executed, one can recognize two layers: "Data Processing" and "Image Processing". These layers can be decoupled.
"Data Processing" includes operations related to managing a database, its state, and the information it stores. This layer is making sure that information about everything that Gradient Image Processing is about can be retrieved and is always valid. It handles information about Users, Client Applications, Configurations, original images information, result images information, and state of individual image processing.
"Image Processing" includes operations related to working with images. Image scaling, compression, crop, storing, and retrieving from adequate storage are parts handled in the "Image Processing" layer. It is imperative to make sure that this layer does not have any operations related to data processing.
Necessary input for the "Image Processing" is provided by the "Data Processing" layer. The "Data Processing" layer updates the database based on outputs received from "Image Processing".
Detailed information about all entities and their attributes can be checked here.
The modules diagram presents all implemented controllers and services. Isolated layers are:
Some of the components in the diagram are marked with different colors, repeated or even skipped being drawn, for the purpose of easier understanding of the diagram. Hence the following notes:
Detailed information about every single service in this diagram, with its methods, signatures, and purpose, can be checked in technical documentation here.
As visible from previous chapters, the service communicates using http and WebSocket protocols.
For http API usage, the OpenAPI specification is provided for download here. It can be visually presented in online tools like swagger.
To utilize communication using WebSocket (for real-time part of API) check usage guidelines here.
Gradient Image Processing can be extended based on specific feature requirements. In this chapter, some of the possible extensions are described.
In the following illustration, one can see an approach where Gradient Image Processing implementation is scaled into multiple independent services each with its own purpose and responsibilities.
By this approach, Gradient Image Processing functionality is separated into the following independent modules:
Modules/Services that are considered stateless can safely be instantiated as more than one running instance, and it does not matter when and under which circumstances are these services executed. Since the logic of these services is pure (analogy with pure functions), it can be executed in concurrency without affecting the final result and product stability.
On the other side, modules that are considered stateful can not be simply instantiated in concurrency mode since the state these are holding is interconnected. Additionally, the availability of requested information from the state would depend on which instance that information is requested.