Category: personal AnGoL

1. Overview Spring Cloud Data Flow is a cloud-native toolkit for building real-time data pipelines and batch processes. Spring Cloud Data Flow is ready to be used for a range of data processing use cases like simple import/export, ETL processing, event streaming, and predictive analytics. In this tutorial, we’ll learn an example of real-time Extract Transform and Load (ETL) using a stream pipeline that extracts data from a JDBC database, transforms it to simple POJOs and loads it into a MongoDB.

Source: ETL with Spring Cloud Data Flow | Baeldung

In the pursuit of data protection, businesses nowadays face more hurdles in the security landscape than ever before. We know there’s a growing demand for reliable, scalable infrastructure, but issues with downtime are complicating businesses’ confidence in their existing systems, implicating all-too-precious data in the process.

Source: Your Data Protection and Disaster Recovery Checklist – DZone Security

One aspect of knowledge transmission which is more difficult to study is the role of the genius. Cultural generation can get stuck in local optima. Only the genius can see over the valley to the mountain. The occasional genius may have been important even in knowledge generation in the pre-science era. In addition, these kinds of cultural evolution processes work best when feedback is quick and clear. Lengthen the time between input and output and all bets are off. Still this peculiar experiment illustrates how much cultural transmission can achieve and how theory can so dominate our thinking that it reduces vital experimentation.

Source: Improving But Not Learning by Doing – Marginal REVOLUTION

• Create mind maps with technical tools. First, you can start using tech tools to visualize your thought processes, gain access to new information, and think more abstractly about the problems you face. For example, you can create interactive mind maps that let you organize different categories of thoughts and related subjects, so you can visualize a high-level rundown of the problem at a glance. From there, it will be easier to spot flaws and brainstorm solutions.
• See problems through others’ eyes. Thinking laterally, or “outside the box,” is the simplest way to think critically. We’re often limited by our own perceptions, experiences, and cognitive biases, and the best way to remove those filters is to see the problem through someone else’s eyes. There are many ways to do this, but the simplest is simply picking someone’s brain about the problem you’re facing—they may bring to light ideas or avenues of thought you hadn’t previously considered.

Source: How to Improve Critical Thinking as a Programmer – DZone Agile

Issues with Asychronous Communication

1. How should a long-running process on the server be presented to the client? If the server is experiencing a high peak in demand, or if the resources are scarce, chances are the client will be affected too. Should it just wait? Should it timeout? How should the client interpret this situation?
2. When an upstream dependency (server A calls server B) is unavailable, should the client care? Why should the client be sensitive to our ecosystem. From its standpoint, it reached the server, the rest is all internals.
3. Guaranteed response time: what if I told you that my server will respond in O(1) no matter what. How is that for usability?
4. What if the client does not need a response right away? Assume that I place my order in the context of an ecommerce application. I know what I ordered, I know how much I paid and I know when and where I will be delivered. Why should I hang in there until the server does all its housekeeping (updating product stocks, notifying the warehouse, ….)
5. Better yet, you don’t know whom you’re talking to but you’re sure that your request will be honored. I always feel this way when I need an administrative paper. It seems that I always need to find the right person, and more importantly, at the right time. I always wished I could just submit my request to a “mailbox” and just head home and be sure that it will be fulfilled.
6. My request is more urgent that yours. Now you’re in a nuclear power plant, and maintenance staff keeps track of the major events there by posting these events to the monitoring application, business as usual, cooling check… Suddenly, Mr Hero realizes that a major event that jeopardizes the safety of the whole neighborhood is about to take place, and posts an emergency stop payload, now I wouldn’t like to be the one who processes a ventilation check while making the emergency stop wait.

Source: Microservices: Why Asynchronous Communications? – DZone Microservices

Why Asynchronous?

Or why stick to synchronous communications should we ask? It is true that synchronous communications has come to be regarded as the de facto pattern to exchange information between any two endpoints. Think of TCP, HTTP, FTP … . It does indeed exhibit the following advantages:

1. Simpler reasoning and tracing: Everything starts when an outbound request in made from the client, and we can step through the whole process all the way until the request is processed by the server and a response is sent accordingly.
2. “Instantaneous” feedback: When a client breaks the contract, or the server deems the current request as invalid, feedback is sent right away to the client. This can take the form of response codes, redirects, or event stack traces.
3. Natural: In the sense that we like to think about thing A happens before thing B, or thing C happens as a consequence of thing B.
4. Translates directly to models: We’re all used to sequence diagrams on UML, and very few people can tell from the back of their head how asynchronous calls are modeled. Instead, we like to think of them as a succession of synchronous calls.

Source: Microservices: Why Asynchronous Communications? – DZone Microservices

For this reason, it is absolutely critical that KRs be specific enough to be measurable (this is where KPIs come in), and that they contain a numerical element that makes them easier to form metrics for. In fact, one technique to ensure KR accuracy is creating KRs according to the S.M.A.R.T. method. The S.M.A.R.T. method dictates that goals should be: Specific; Measurable; Achievable; Relevant; and Timebound.

Source: Trillion-Dollar Teamwork: Goal-Setting With OKRs – DZone Agile

If event time is not relevant and latencies in the seconds range are acceptable, Spark is the first choice. It is stable and almost any type of system can be easily integrated. In addition it comes with every Hadoop distribution. Furthermore, the code used for batch applications can also be used for the streaming applications as the API is the same.

Source: Spark Streaming vs. Kafka Streaming – DZone Big Data

The Long Game
The long game is the opposite of the short game, it means paying a small price today to make tomorrow’s tomorrow easier. If we can do this long enough to see the results, it feeds on itself.

Source: The Surprising Power of The Long Game

In this article, we will discuss different ways of running Spring Boot applications:
1. Running from an IDE
2. Running as a Packaged Application
3. Using the Maven Plugin
4. Using External Tomcat
5. Using the Gradle Plugin

Source: 5 Ways to Run Spring Boot Apps – DZone Java