3 posts tagged with "web"

The purpose of this article is to explain the architecture and the technology stack used in the creation of the Eco-system that was delivered to a Fortune 500 public retail company. Due to NDA agreements, I’m obligated to obscure the actual purposes and specific use cases of those applications.

Background​

Our company contracted with this Fortune 500 public retail company company, and let’s refer to it as X from now, to build a data collection application of some type (beyond the scope of this article). That application was built in a Monolithic architecture. Meaning, One application, one project that connects all the application resources, serves the web pages, handles all the HTTP and AJAX requests. The application would also interface with the databases, caching services, ActiveDirectory, and anything else and the application may utilize. For that time and given that it was only application and we weren’t envisioning a development for a whole eco-system. And Honestly, it was my very first project in Node.js or anything for that matter and I don’t think I even know what a microservices would even mean. 🤔 😗

The first database design was consisting of about 16 tables. (I also can’t share that) but it’s worth mentioning that I designed that database as I was taking Database 1 and that was my first ever database design. I even remember I would go to class and come back and make more changes (Just like that). The current database was redesigned from the ground-up to to accommodate the new applications. It was also better normalized and optimized by the addition of multiple indexes to improve the performance of the queries. Now it’s 50 tables and 4 views.

Fast Forward ...

Apparently the company X liked what we did and they came back to us with 4 new projects and V2.0 of the the collection application with more functionality, business rule and administrative capabilities. Then we had to rethink, the architectural choices that we made before. Including the database design I was actually done with Database I and Database II then

System Architecture​

The current architecture consists of 11 Microservices, connected together through a unifying RESTful API. All the applications and mobile apps communicate with the system resources and with each other through that API. The microservices, depending on their purposes and functionality, would interact with one or more of the following pieces (The database server (Microsoft SQL Server), the Caching Service/ Session Manager (Redis), ActiveDirectory (User and Domain Management), Queueing service (RabbitMQ), Machine Learning and data processing service (R servers), Reporting Service (CrystalReports) .

Server architecture​

In this project, we used 6 servers; 3 development servers located and managed by our office and 3 servers for Staging and Production Servers that are managed by the client but we have access to do some debugging and deployment to their staging servers.

I was responsible for managing and development servers and configuring the Continuous Integration CI builds. And the Lead Developer was responsible for deploying our code to their staging servers.

Technology Stack:​

• Node.js / Restify
• Microsoft SQL Server
• Bookshelf.js / Knex
  • The micro service uses an Object Relational Mapping Module for data access to abstract away the vendor-specific SQL syntax which makes the application more portable and more easily-switchable to any SQL flavor (MSSQL, MySQL, PostgreSQL, … etc.)
    • we used to use Sequelize but we switched over to bookshelf/Knex as it gives us more control over the queries and at the time it seemed to be much easier.
• Redis
• RabbitMQ
• DeployR / R Server
• ActiveDirectory
• Drone.io

Development Style​

The development was intended to, and did start, in a Test-Driven-Development TDD. But I confess, we didn't follow through leading to a tremendous technical debt. A problem that we started to slowly reduce, by writing more unit tests and by utilizing applications such as PAW. Which allows us to create collections of HTTP requests with its parameters and run them all it once making it easy to spot any failures to quickly go fix it.

We also started a process to review and accept any features and bug fixes; the project manager is the only person who's allowed to close any issue or task item. Though this might not be the best solution, but it's working for the time being until we pay our debt.

In this post, I'll demonstrate 2 ways to perform custom sorting. It all happened when a client asked to sort a lookup table/dropdown in a specific order. S -> U -> N

Example:​

Statuses:

|id |     name      |
|---|:-------------:|
| 1 | Complete (S)  |
| 2 |Dropped out (U)|
| 3 |in progress (N)|
| 4 |Lost Funds (U) |

Recommended: If you have access to the database.​

Usually what I'd do is adding a sort_order column to the table in the database and use it when querying the records. As such,

|id |     name      | sort_order |
|---|:-------------:|:---------- |
| 1 | Complete (S)  |      1     |
| 2 |Dropped out (U)|      2     |
| 3 |in progress (N)|      4     |
| 4 |Lost Funds (U) |      3     |

and the query would simply be:

    SELECT *
    FROM statuses
    ORDER_BY sort_order

This is obviosly the cleanest and the recommended way to approach this problem.

However, in this particular case, we weren't at liberty to make changes to the database design. So, we had to discuss performing the sort at the query time with some other custom methods, OR processing it using JS (The application in question is Node.JS/Angular Application).

Solution1: Custom Order using SQL​

The first Solution involved Regular expressions to extract the last bits of the names (S), (N), (U) Which are what will be used in the sorting

SELECT *
FROM statuses
ORDER BY 
    CASE substring(name from '\([A-Z]\)')
        WHEN '(S)' THEN 1
        WHEN '(U)' THEN 2
        WHEN '(N)' THEN 3
        ELSE 5
    END

Custom Order by processing the data in JS​

Given that we already had a specific criteria for the sort (S -> U -> N), I created a variable sorter of those 3 values along with the corresponding desired order. Javascript sort function allows for custom comparing functions to be used in the sorting process. So, we will be creating this comparing function compare(). What this function basically does, is that it uses the same regular expression to extract the bit with the parantheses, For each element it finds the correspondoing order from the sorter variable and uses that as a comparer.

var compare = function(a, b) {
  var sorter = [{value: "(S)", order: 1}, {value: "(U)", order: 2}, {value: "(N)", order: 3}]

  var a_status = a.name.match(/\([A-Z]\)/g)
  var b_status = b.name.match(/\([A-Z]\)/g)
  
  return sorter.find(function(element) { return element.value == a_status[0]; }).order - sorter.find(function(element) { return element.value == b_status[0]; }).order
}

var sorted = release.sort(compare)
console.log(sorted)

Eventually we opted to the second solution. We use an ORM Node Module and making the custom order in sql wouldn't have been as easy to implement.

Confession:​

I totally forgot about adding the sort_order Column until the time I wrote this blog. But again we weren't have been able to use it anyway. Cudos to Rubber Ducking!!!

One small issue I faced while I was working on my portfolio website, was that I wanted the project modal to be able to open each other. For example. I wanted the portal modal to be able to reference all the other applications and API modals that it’s linked to. The problem was a little compound.

do I want to close the current modal before opening the other modal? Or do I want to stack the modals on top of each other?

Approach 1​

What I first found as a solution was to make use of the two tags that bootstrap offers for modals data-dismiss="modal" and data-toggle=“modal”.

<a href="#" data-dismiss="modal" data-toggle="modal">This is the link to the other modal</a>

This worked fine. It closes the portal modal and opens the next modal. BUT for some reason, the second modal wouldn’t scroll. And since the modal background was a little transparent, I could see the page behind the modal scrolling 🤔

Approach 2​

Another fast an easy solution was to use JQuery to toggle the modal.

<a id="modalTogglerBtn">This is the link to the other modal</a>

$("modalTogglerBtn").on("click", function(){
    $('#myModal').modal('toggle');
});

This did the trick. Not really. The new modal would open and it would scroll. So what was the problem? As I close the modal on top of the stack. The same problem of not being able to scroll on the modal would happen to the first modal.(The one down the stack).

Here’s the problem then, something with the modal messes the scrolling up and transfer it from the modals back to the body.

Approach 3 (The Solution) 🎉​

So here’s what I found out. Bootstrap, upon the creation, adds a modal-open class to the body tag. This class is what causes the scrolling behavior to be focused in the modals rather than the body itself. As we would close the top modal, bootstrap would remove that class from the body tag. That's why neither of the first approaches worked. So we need a why or a mechanism to check for open modals first before removing that tag.

$(document).on('hidden.bs.modal', function (event) {
  if ($('.modal:visible').length) {
    $('body').addClass('modal-open');
  }
});

The hidden.bs.modal is invoked when the modal is fully hidden (after CSS transitions have completed) Ref.JavaScript · Bootstrap . Modal Events