Codeaches

Setup Kubernetes Cluster on Google Cloud Platform (GCP) and connect using Cloud SDK

2020-01-07T00:00:00-07:00

In this guide, let’s create a Kubernetes cluster using Google Cloud Platform (GCP) account and connect to this cluster using GCP Cloud SDK.

Create a Google Cloud Platform (GCP) account
1. Sing up for Google Cloud Platform.
2. Create a new project
Create a new Kubernetes cluster on GCP
Download and configure Google Cloud SDK on local PC
Connect to GCP Kubernetes cluster using Google Cloud SDK
Kubernetes commandline setup
Try out basic kubernetes commands

Create a Google Cloud Platform (GCP) account

Sing up for Google Cloud Platform.

The first step is to create an account in Google Cloud Platform. Log into Google Cloud Platform using your Gmail/Google account credentials here.

Create a new project

Traverse to Kubernetes Engine > Clusters and create a new Project.

Create a new Project

Here, I have created a new project called codeaches-project.

I have opted for a free trial to start using Kubernetes Engine.

Create a new Kubernetes cluster on GCP

Once, you subscribe to free trial, traverse to Kubernetes Engine > Clusters and click on Create Cluster to create a new kubernetes cluster.

I have opted for default values where we get 3 nodes in a cluster.

Now we have a kubernetes cluster ready for us!

At this point, we have a cluster codeaches-cluster-1 in our project codeaches-project on Google Cloud Platform. Let’s move on to downloading the Google Cloud SDK on our local PC which will help us with connecting to the above created cluster.

Download and configure Google Cloud SDK on local PC

Download the Google Cloud SDK installer using the link here.

Once the installation is complete, open the command prompt and vefiry that the Cloud SDK is setup.

> gcloud version

Output

Google Cloud SDK 275.0.0
bq 2.0.51
core 2020.01.03
gsutil 4.46

Connect to GCP Kubernetes cluster using Google Cloud SDK

Use the gcloud init command to perform several common SDK setup tasks. These include authorizing the SDK tools to access Google Cloud Platform using your user account credentials and setting up the default SDK configuration.

> gcloud init

Output

Welcome! This command will take you through the configuration of gcloud.

Your current configuration has been set to: [default]

You can skip diagnostics next time by using the following flag:
  gcloud init --skip-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).

You must log in to continue. Would you like to log in (Y/n)?  y

Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?....................


You are logged in as: [mail@somedomain.com].

Pick cloud project to use:
 [1] secret-argon-123456
 [2] Create a new project
Please enter numeric choice or text value (must exactly match list
item):  1

Your current project has been set to: [secret-argon-123456].

Note that the project ID secret-argon-123456 corresponds to the earlier created project with name codeaches-project. secret-argon-123456 is a random ID assigned by Google Cloud Platform.

> gcloud projects list

Output

PROJECT_ID           NAME               PROJECT_NUMBER
secret-argon-123456  codeaches-project  654321123456

Kubernetes commandline setup

Earlier, we had created a kubernetes cluster named codeaches-cluster-1 using the Google Cloud Platform UI.

To access the Kubernetes Engine cluster, we need kubernetes commandline. Let’s install the kubectl.

> gcloud components install kubectl

Once, kubectl is installed successfully, configure the local kubectl to point to codeaches-cluster-1 on GCP.

> gcloud container clusters get-credentials codeaches-cluster-1 --zone us-central1-a --project secret-argon-123456

Output

Fetching cluster endpoint and auth data.
kubeconfig entry generated for codeaches-cluster-1.

Now you are all set to use the locally installed Cloud SDK to interact with Kubernetes Cluster on GCP

Try out basic kubernetes commands

Get cluster details

> kubectl cluster-info

Output

Kubernetes master is running at https://33.211.100.10
GLBCDefaultBackend is running at https://33.211.100.10/api/v1/namespaces/kube-system/services/default-http-backend:http/proxy
Heapster is running at https://33.211.100.10/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://33.211.100.10/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://33.211.100.10/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Get node details

> kubectl get nodes

Output

NAME                                                 STATUS   ROLES    AGE     VERSION
gke-codeaches-cluster-1-default-pool-38905352-080c   Ready    <none>   8m31s   v1.12.8-gke.10
gke-codeaches-cluster-1-default-pool-38905352-km5p   Ready    <none>   8m31s   v1.12.8-gke.10
gke-codeaches-cluster-1-default-pool-38905352-mk4c   Ready    <none>   8m31s   v1.12.8-gke.10

Summary

This concludes our guide to creating creating Kubernetes cluster on GCP and connecting to it through Cloud SDK running on our local machine!

Your feedback is always appreciated. Happy coding!

Integrate embedded Apache ActiveMQ 5 (Classic) JMS Broker with Spring Boot application

2020-01-06T00:00:00-07:00

Apache ActiveMQ is the most popular open source JMS server. ActiveMQ supports Spring for configuration of the JMS client side as well as for configuring the JMS Message Broker.

In this post, let’s integrate an embedded ActiveMQ JMS 5 broker (Classic version as they call it) with a simple Spring Boot application which sends/recieves data from the embedded ActiveMQ broker.

Create spring boot starter application
Configure additional project dependencies in pom.xml
Configure Embedded ActiveMQ Broker
Configure JMS Producer
Configure JMS Consumer
Test JMS producer and consumer

Create spring boot starter application

Building the bare bone Spring Boot Service is simple when Spring Initializr is used. Spring Initializr generates spring boot project with just what you need to start quickly! Let’s start off with one.

Create a Spring Boot starter project using Spring Initializr

Let’s utilize the pre-configured Spring Initializr which is available here to create embedded-activemq-5-jms-broker starter project.

Click on Generate Project. This downloads a zip file containing embedded-activemq-5-jms-broker project. Import the project to your IDE.

Configure additional project dependencies in pom.xml

We need an additional activemq-broker dependency which will give all the classed needed for setting up an embedded ActiveMQ JMS 5 broker.

Uptate pom.xml with below dependencies. Here we are adding activemq-pool dependency as well to create a pooled connection factory.

pom.xml

<dependency>
	<groupId>org.apache.activemq</groupId>
	<artifactId>activemq-broker</artifactId>
</dependency>

<dependency>
	<groupId>org.apache.activemq</groupId>
	<artifactId>activemq-pool</artifactId>
</dependency>

Let’s add broker URL in application.properties. In this guide, we shall embed an active mq broker to run on port 61616. We also need a queue to send/recieve messages. We shall configure this queue name as well here.

src/main/resources/application.properties

activemq.broker.url=tcp://0.0.0.0:61616
activemq.queue.name=my-queue-1

Configure Embedded ActiveMQ Broker

Create an Active MQ embedded broker as shown below. This broker will be started when the application starts up.

com.codeaches.activmq.embedded.JmsConfig.java

@Configuration
public class JmsConfig {

  @Value("${activemq.broker.url}")
  String brokerUrl;

  @Bean
  public BrokerService broker() throws Exception {

    BrokerService broker = new BrokerService();
    broker.setPersistent(false);
    broker.setUseJmx(true);
    broker.addConnector(brokerUrl);
    return broker;
  }
}

Configure JMS Producer

Let’s use Spring’s JmsTemplate to send messages to the embedded broker. We shall also use a pooled connection factory for our JmsTemplate as shown below.

Update JmsConfig.java with JmsTemplate bean.

com.codeaches.activmq.embedded.JmsConfig.java

  @Bean
  public JmsTemplate jmsTemplate() {
    return new JmsTemplate(new PooledConnectionFactory(brokerUrl));
  }

Create a producer class which uses the above created JmsTemplate to send the data to embedded Active MQ server.

com.codeaches.activmq.embedded.JmsProducer.java

@Service
public class JmsProducer {

  Logger log = LoggerFactory.getLogger(JmsProducer.class);

  @Autowired
  private JmsTemplate jmsTemplate;

  @Value("${activemq.queue.name}")
  String destination;

  public void send(String message) {
    jmsTemplate.convertAndSend(destination, message);
    log.info("Sent message='{}'", message);
  }
}

Configure JMS Consumer

Let’s use Spring’s JmsListener to read the messages from the JMS. JmsListener needs a connection factory. We shall use pooled connection factory in the consumer as well.

Update JmsConfig.java with DefaultJmsListenerContainerFactory bean.

com.codeaches.activmq.embedded.JmsConfig.java

  @Bean
  public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {

    DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
    factory.setConnectionFactory(new PooledConnectionFactory(brokerUrl));
    return factory;
  }

Create a consumer class which uses Spring’s JmsListener to recieve the data from embedded Active MQ server.

com.codeaches.activmq.embedded.JmsConsumer.java

@Service
public class JmsConsumer {

  Logger log = LoggerFactory.getLogger(JmsConsumer.class);

  @JmsListener(destination = "${activemq.queue.name}")
  public void receive(String message) {
    log.info("Received message='{}'", message);
  }
}

Now that we have both producer and consumer, let’s build a simple webservice method which uses earlier created JMS producer to send data to the queue my-queue-1 as shown below.

Here, the sendDataToJms(message) webservice method forwards the message sent by the caller to JMS queue.

com.codeaches.activmq.embedded.MyRestController.java

@RestController
public class MyRestController {

  @Autowired
  JmsProducer jmsProducer;

  @PostMapping("/send")
  public void sendDataToJms(@RequestParam String message) {
    jmsProducer.send(message);
  }
}

Start the application

Tomcat console log:

JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
Using Persistence Adapter: MemoryPersistenceAdapter
Apache ActiveMQ 5.15.10 (localhost, ID:DESKTOP-MYPCNAME-53882-1325267078594-0:1) is starting
Listening for connections at: tcp://DESKTOP-MYPCNAME:61616
Connector tcp://DESKTOP-MYPCNAME:61616 started
Apache ActiveMQ 5.15.10 (localhost, ID:DESKTOP-MYPCNAME-53882-1325267078594-0:1) started
For help or more information please see: http://activemq.apache.org
Initializing ExecutorService 'applicationTaskExecutor'
Tomcat started on port(s): 8080 (http) with context path ''
Started EmbeddedJmsBrokerApplication in 2.17 seconds (JVM running for 3.17)

Test JMS producer and consumer

Let’s invoke the webservice by sending a hello message. JmsProducer would send the message and JmsConsumer will consume the hello message.

curl -i -X POST http://localhost:8080/send?message=hello

Tomcat Log

INFO com.codeaches.activmq.embedded.JmsProducer   : Sent message='hello'
INFO com.codeaches.activmq.embedded.JmsConsumer   : Received message='hello'

Summary

This concludes our guide to integrating an embedded Apache ActiveMQ JMS with a simple Spring Boot application.

Your feedback is always appreciated. Happy coding!

Create a stream on Pivotal Cloud Foundry (PCF) using Spring Cloud Data Flow (SCDF) server

2020-01-03T00:00:00-07:00

In this tutorial, let’s use Spring provided HTTP source and LOG sink applications to create a stream and deploy it to cloudfoundry. We shall use SCDF server running locally on our machine to deploy the stream to cloudfoundry.

Prerequisites
Start SCDF Shell CLI on your PC
Register the out-of-the-box http and log spring boot apps
Create the stream
Deploy the stream
Validate the deployed stream
Test httpLogStream stream using PCF CLI

Prerequisites

An account on Pivotal Cloud Foundry (PCF). You can create one here.
PCF Command Line Interface (CLI) installed on your computer. PCF CLI can be found in tools section of your PCF account.
An up and running SCDF server on PCF. You can follow the tutorial here to install SCDF server on PCF.
Download SCDF CLI jar to your PC. You can follow the tutorial here to download.

Start SCDF Shell CLI on your PC

java -jar spring-cloud-dataflow-shell-2.3.0.RELEASE.jar --dataflow.uri=https://my-codeaches-scdf-server.cfapps.io --dataflow.username=user001 --dataflow.password=pass001 --skip-ssl-validation=true

Output

  ____                              ____ _                __
 / ___| _ __  _ __(_)_ __   __ _   / ___| | ___  _   _  __| |
 \___ \| '_ \| '__| | '_ \ / _` | | |   | |/ _ \| | | |/ _` |
  ___) | |_) | |  | | | | | (_| | | |___| | (_) | |_| | (_| |
 |____/| .__/|_|  |_|_| |_|\__, |  \____|_|\___/ \__,_|\__,_|
  ____ |_|    _          __|___/                 __________
 |  _ \  __ _| |_ __ _  |  ___| | _____      __  \ \ \ \ \ \
 | | | |/ _` | __/ _` | | |_  | |/ _ \ \ /\ / /   \ \ \ \ \ \
 | |_| | (_| | || (_| | |  _| | | (_) \ V  V /    / / / / / /
 |____/ \__,_|\__\__,_| |_|   |_|\___/ \_/\_/    /_/_/_/_/_/

2.3.0.RELEASE

Welcome to the Spring Cloud Data Flow shell. For assistance hit TAB or type "help".
Successfully targeted https://my-codeaches-scdf-server.cfapps.io
dataflow:>

Register the out-of-the-box `http` and `log` spring boot apps

Cloudfoundry provides with few out-of-the-box source and sink spring boot applications which can be used for stream creation. Lets register the out-of-the-box http and log spring boot apps, specific to rabbit messaging broker, in SCDF server.

Register HTTP Source Spring Boot App

dataflow:>app register --name http --type source --uri maven://org.springframework.cloud.stream.app:http-source-rabbit:2.1.1.RELEASE

Output

Successfully registered application 'source:http'

Register LOG Sink Spring Boot App

dataflow:>app register --name log --type sink --uri maven://org.springframework.cloud.stream.app:log-sink-rabbit:2.1.2.RELEASE

Output

Successfully registered application 'sink:log'

Create the stream

Let’s utilize the above registered apps http and log to create http|log stream using stream create command on SCDF shell terminal.

This stream will take HTTP POST request and prints the body in log file.

dataflow:>stream create --name httpLogStream --definition "http | log"

Output

Created new stream 'httpLogStream'

Deploy the stream

dataflow:>stream deploy httpLogStream --platformName dev

Output

Deployment request has been sent for stream 'httpLogStream'

Validate the deployed stream

dataflow:>stream validate 'httpLogStream'

Output

╔═════════════╤═════════════════╗
║ Stream Name │Stream Definition║
╠═════════════╪═════════════════╣
║httpLogStream│http | log       ║
╚═════════════╧═════════════════╝


httpLogStream is a valid stream.
╔═══════════╤═════════════════╗
║ App Name  │Validation Status║
╠═══════════╪═════════════════╣
║source:http│valid            ║
║sink:log   │valid            ║
╚═══════════╧═════════════════╝

dataflow:>stream info 'httpLogStream'

Output

╔═════════════╤═════════════════╤═══════════╤════════╗
║ Stream Name │Stream Definition│Description│ Status ║
╠═════════════╪═════════════════╪═══════════╪════════╣
║httpLogStream│http | log       │           │deployed║
╚═════════════╧═════════════════╧═══════════╧════════╝

Stream Deployment properties: {
  "log" : {
    "resource" : "maven://org.springframework.cloud.stream.app:log-sink-rabbit:jar",
    "spring.cloud.deployer.group" : "httpLogStream",
    "version" : "2.1.2.RELEASE"
  },
  "http" : {
    "resource" : "maven://org.springframework.cloud.stream.app:http-source-rabbit:jar",
    "spring.cloud.deployer.group" : "httpLogStream",
    "version" : "2.1.1.RELEASE"
  }
}

Test `httpLogStream` stream using PCF CLI

Log into PCF using the cf login command.

$ cf login -a api.run.pivotal.io -u "you@some-domain.com" -p "yourpassword"  -o "your-org" -s "dev"

Output

API endpoint: api.run.pivotal.io
Authenticating...
OK

Targeted org your-org
Targeted space dev

API endpoint:   https://api.run.pivotal.io (API version: 2.138.0)
User:           you@some-domain.com
Org:            your-org
Space:          dev

Once the stream creation and deployment is successful, PCF creates random routes (urls) for both log and sink applications which can be validated using cf apps command

$ cf apps

Output

Getting apps in org your-org / space dev as you@some-domain.com...
OK

name                            requested state   instances   memory   disk   urls
NEe1JMP-httpLogStream-http-v1   started           1/1         2G       1G     NEe1JMP-httpLogStream-http-v1.cfapps.io
NEe1JMP-httpLogStream-log-v1    started           1/1         2G       1G     NEe1JMP-httpLogStream-log-v1.cfapps.io

Post a sample message to the stream

Post a sample hello world message to http application using the route NEe1JMP-httpLogStream-http-v1.cfapps.io as shown below. The message will be picked up by http app and passed to log application.

$ curl -i -H "Content-Type:application/text" -X POST -d 'hello world' https://NEe1JMP-httpLogStream-http-v1.cfapps.io

Output

HTTP/1.1 202 Accepted
Date: Sat, 04 Jan 2020 01:58:19 GMT
X-Vcap-Request-Id: c94b6a06-f3e2-4982-7f4b-34fbe3664713
Content-Length: 0
Connection: keep-alive

Once the message is posted successfully, hello world will be printed in the logs of log application.

Tail the log of log application using cf logs command

Tail the log of data-flow-server-hd6lIb0-httpLogStream-log application using cf logs command.

cf logs --recent NEe1JMP-httpLogStream-log-v1

Output

[httpLogStream-1] NEe1JMP-httpLogStream-log-v1             : hello world

Summary

Congratulations! You just deployed a stream to cloudfoundry using Spring Cloud Dataflow Server (SCDF).

Your feedback is always appreciated. Happy coding!

Install Spring Cloud Data Flow (SCDF) server and Skipper Server on Pivotal Cloud Foundry (PCF)

2020-01-03T00:00:00-07:00

In this tutorial, let’s configure and deploy spring cloud data flow server to pivotal cloudfoundry (PCF). For deploying streams, the Data Flow Server delegates the deployment work to Skipper Server. Hence, we shall configure and deploy Skipper server as well.

Prerequisites
Provision services needed for Skipper server
Deploy Skipper server
Provision services needed for SCDF server
Deploy SCDF server
Setup SCDF Shell CLI on your PC
Setup Skipper Shell CLI on your PC

Prerequisites

An account on Pivotal Cloud Foundry (PCF). You can create one here
PCF Command Line Interface (CLI) installed on your computer. PCF CLI can be found in tools section of your PCF account

Provision services needed for Skipper server

Log into your PCF account using cf command

$ cf login -a api.run.pivotal.io -u "you@some-domain.com" -p "yourpassword" -o "your-org" -s "util"

API endpoint: api.run.pivotal.io
Authenticating...
OK
Targeted org your-org
Targeted space util

API endpoint:   https://api.run.pivotal.io (API version: 2.128.0)
User:           you@some-domain.com
Org:            your-org
Space:          util

Provision Postgresql

Skipper server uses an RDBMS to store state. For our example, we shall use Postgresql. Schemas on Postgresql DB will be created on Skipper server startup.

cf create-service elephantsql turtle my_postgresql

Output

Creating service instance my_postgresql in org your-org / space util as you@some-domain.com...
OK

Deploy Skipper server

Download Skipper server

wget https://repo.spring.io/release/org/springframework/cloud/spring-cloud-skipper-server/2.2.1.RELEASE/spring-cloud-skipper-server-2.2.1.RELEASE.jar

Configure Skipper properties

skipper-server-manifest.yml

---
applications:
- name: skipper-server
  random-route: true
  memory: 1G
  disk_quota: 1G
  instances: 1
  path: spring-cloud-skipper-server-2.2.1.RELEASE.jar
  buildpack: java_buildpack_offline
  routes:
  - route: my-codeaches-skipper-server.cfapps.io
  env:
    JBP_CONFIG_OPEN_JDK_JRE: '{ jre: { version: 13.+ } }'
    SPRING_APPLICATION_NAME: skipper-server
    SPRING_PROFILES_ACTIVE: cloud
    JBP_CONFIG_SPRING_AUTO_RECONFIGURATION: '{enabled: false}'
    SPRING_CLOUD_SKIPPER_SERVER_ENABLE_LOCAL_PLATFORM: false

    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_CONNECTION_URL: https://api.run.pivotal.io
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_CONNECTION_ORG: your-org
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_CONNECTION_SPACE: dev
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_DEPLOYMENT_DOMAIN: cfapps.io
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_CONNECTION_USERNAME: you@some-domain.com
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_CONNECTION_PASSWORD: yourpassword
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_CONNECTION_SKIP_SSL_VALIDATION: true
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_DEPLOYMENT_DELETE_ROUTES: false
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_DEPLOYMENT_SERVICES: my_rabbit
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_DEPLOYMENT_STREAM_ENABLE_RANDOM_APP_NAME_PREFIX: false
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[dev]_DEPLOYMENT_MEMORY: 2048m

    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_CONNECTION_URL: https://api.run.pivotal.io
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_CONNECTION_ORG: your-org
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_CONNECTION_SPACE: qa
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_DEPLOYMENT_DOMAIN: cfapps.io
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_CONNECTION_USERNAME: you@some-domain.com
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_CONNECTION_PASSWORD: yourpassword
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_CONNECTION_SKIP_SSL_VALIDATION: true
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_DEPLOYMENT_DELETE_ROUTES: false
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_DEPLOYMENT_SERVICES: my_rabbit
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_DEPLOYMENT_STREAM_ENABLE_RANDOM_APP_NAME_PREFIX: false
    SPRING_CLOUD_SKIPPER_SERVER_PLATFORM_CLOUDFOUNDRY_ACCOUNTS[qa]_DEPLOYMENT_MEMORY: 2048m
  services:
  - my_postgresql

Push Skipper server to cloudfoundry using PCF CLI

cf push -f skipper-server-manifest.yml

Output

Pushing from manifest to org your-org / space util as you@some-domain.com...
Using manifest file M:\scdf-basics\skipper-server-manifest.yml
...
...
Waiting for app to start...

name:              skipper-server
requested state:   started
routes:            my-codeaches-skipper-server.cfapps.io
...
...
     state     since                  cpu      memory         disk           details
#0   running   2020-01-03T04:39:02Z   166.0%   280.8M of 1G   228.7M of 1G

Provision services needed for SCDF server

Provision Redis

Since analytics feature is enabled by default, the Data Flow server is expected to have a valid Redis store available as analytic repository.

cf create-service rediscloud 30mb my_redis

Output

Creating service instance my_redis in org your-org / space util as you@some-domain.com...
OK

Provision MySQL

A relational database is used to store stream and task definitions as well as the state of executed tasks. For our example, we shall use MySQL server.

M:\scdf-basics>cf create-service cleardb spark my_mysql

Output

Creating service instance my_mysql in org your-org / space util as you@some-domain.com...
OK

Deploy SCDF server

Download Data Flow server

wget https://repo.spring.io/release/org/springframework/cloud/spring-cloud-dataflow-server/2.3.0.RELEASE/spring-cloud-dataflow-server-2.3.0.RELEASE.jar

Configure Data Flow properties

dataflow-server-manifest.yml

---
applications:
- name: dataflow-server
  random-route: true
  memory: 2G
  disk_quota: 2G
  instances: 1
  path: spring-cloud-dataflow-server-2.3.0.RELEASE.jar
  routes:
  - route: my-codeaches-scdf-server.cfapps.io
  env:
    SPRING_APPLICATION_NAME: dataflow-server
    JBP_CONFIG_OPEN_JDK_JRE: '{ jre: { version: 13.+ } }'

    SPRING_PROFILES_ACTIVE: cloud
    JBP_CONFIG_SPRING_AUTO_RECONFIGURATION: '{enabled: false}'

    SPRING_CLOUD_SKIPPER_CLIENT_SERVER_URI: https://my-codeaches-skipper-server.cfapps.io/api

    SPRING_APPLICATION_JSON: '{"maven":{"remote-repositories":{"repo1":{"url":"https://repo.spring.io/libs-release"},"repo2":{"url":"https://oss.sonatype.org/content/repositories/snapshots"},"repo3":{"url":"https://oss.sonatype.org/content/repositories/releases"}}}}'

    security.basic.enabled: true
    security.user.name: user001
    security.user.password: pass001
    security.user.role: VIEW,CREATE,MANAGE

  services:
    - my_mysql
    - my_redis

Push Data Flow to cloudfoundry using PCF CLI

cf push -f dataflow-server-manifest.yml

Output

Pushing from manifest to org your-org / space util as you@some-domain.com...
Using manifest file M:\scdf-basics\dataflow-server-manifest.yml
...
...
Waiting for app to start...

name:              dataflow-server
requested state:   started
routes:            my-codeaches-scdf-server.cfapps.io
...
...
     state     since                  cpu      memory         disk           details
#0   running   2020-01-03T04:41:41Z   148.3%   343.8M of 2G   230.3M of 2G

Setup SCDF Shell CLI on your PC

Download SCDF Command Line Interface (CLI)

wget https://repo.spring.io/release/org/springframework/cloud/spring-cloud-dataflow-shell/2.3.0.RELEASE/spring-cloud-dataflow-shell-2.3.0.RELEASE.jar

Start the SCDF Shell CLI

java -jar spring-cloud-dataflow-shell-2.3.0.RELEASE.jar --dataflow.uri=https://my-codeaches-scdf-server.cfapps.io --dataflow.username=user001 --dataflow.password=pass001 --skip-ssl-validation=true

Output

  ____                              ____ _                __
 / ___| _ __  _ __(_)_ __   __ _   / ___| | ___  _   _  __| |
 \___ \| '_ \| '__| | '_ \ / _` | | |   | |/ _ \| | | |/ _` |
  ___) | |_) | |  | | | | | (_| | | |___| | (_) | |_| | (_| |
 |____/| .__/|_|  |_|_| |_|\__, |  \____|_|\___/ \__,_|\__,_|
  ____ |_|    _          __|___/                 __________
 |  _ \  __ _| |_ __ _  |  ___| | _____      __  \ \ \ \ \ \
 | | | |/ _` | __/ _` | | |_  | |/ _ \ \ /\ / /   \ \ \ \ \ \
 | |_| | (_| | || (_| | |  _| | | (_) \ V  V /    / / / / / /
 |____/ \__,_|\__\__,_| |_|   |_|\___/ \_/\_/    /_/_/_/_/_/

2.3.0.RELEASE

Welcome to the Spring Cloud Data Flow shell. For assistance hit TAB or type "help".
Successfully targeted https://my-codeaches-scdf-server.cfapps.io
dataflow:>

Setup Skipper Shell CLI on your PC

wget https://repo.spring.io/release/org/springframework/cloud/spring-cloud-skipper-shell/2.2.1.RELEASE/spring-cloud-skipper-shell-2.2.1.RELEASE.jar

Start the Skipper Shell CLI

java -jar spring-cloud-skipper-shell-2.2.1.RELEASE.jar --spring.cloud.skipper.client.serverUri=https://my-codeaches-skipper-server.cfapps.io/api

Output

skipper:>

Summary

Congratulations! You just deployed both Skipper Server and SCDF Server to Pivotal cloudfoundry (PCF).

Your feedback is always appreciated. Happy coding!

Create and deploy Spring Cloud Config Client on cloudfoundry (PCF)

2020-01-02T00:00:00-07:00

Config Server for Pivotal Cloud Foundry (PCF) is an externalized configuration service, which gives us with a central place to manage an application’s external properties across all environments.

Spring cloud config client applications can use Config Server to manage configurations across environments.

In this tutorial, let’s deploy a sample spring cloud config client application (say..Pet Store App) which binds to Config Server.

Prerequisites
Create spring boot starter application
Deploy the application to PCF
Test
Summary

Prerequisites

An account on Pivotal Cloud Foundry (PCF). You can create one here.
PCF Command Line Interface (CLI) installed on your computer. PCF CLI can be found in tools section of your PCF account.
For this tutorial I will be using cloudfoundry config service named my-config-server. You can refer the guide here to cregister the config service.
my-config-server is registered with the properties file which is in GIT.

Create spring boot starter application

Create a Spring Boot starter project using Spring Initializr

Let’s utilize the pre-configured Spring Initializr which is available here to create pcf-config-client-pet-service starter project.

Click on Generate Project. This downloads a zip file containing pcf-config-client-pet-service project. Import the project to your IDE.

Add a Rest Controller to read the value from properties file

Let’s add a rest controller class which exposes a HTTP GET method pet(). Here, the pet() method returns the value of the variable pet which is configured to dog in dev environement.

com.codeaches.pet.MyController.java

@RestController
@RefreshScope
public class MyController {

  @Value("${pet}")
  public String pet;

  @GetMapping("/pet")
  public String pet() {
    return String.format("My pet is %s", pet);
  }
}

Add a rest controller to read data from json file

Let’s Update the rest controller class with another GET service petDetails(). Here, the petDetails() returns the contents of petDetails.json file of dev environment.

com.codeaches.configclientpcfapp.JSONFileDemoController.java

  @Autowired
  private PlainTextConfigClient configClient;

  @GetMapping("/petDetails")
  public String petDetails() throws IOException {

    try (InputStream input = configClient.getConfigFile("petDetails.json").getInputStream()) {
      return StreamUtils.copyToString(input, Charset.defaultCharset());
    }
  }

Deploy the application to PCF

Create manifest.yml file for deployment

Let’s create manifest.yml file and specify the configuration details like JRE version, path of the application jar file, properties files being referenced as shown below. This file will be used for deployment to PCF.

---
applications:
- name: pet-service
  random-route: true
  instances: 1
  path: target/pcf-config-client-pet-service-0.0.1-SNAPSHOT.jar
  memory: 1G
  env:
    spring.application.name: petDetails
    JBP_CONFIG_OPEN_JDK_JRE: '{ jre: { version: 13.+ } }'
  services:
    - my-config-server

path:
spring.application.name:
services:

Deploy Pet Store App to cloudfoundry

Deploy pcf-config-client-pet-service-0.0.1-SNAPSHOT.jar to PCF using the cf push command.

$ cf push -f manifest.yml

Output

Pushing from manifest to your-org / space dev as you@some-domain.com...
Using manifest file M:\pcf-config-client-pet-service\manifest.yml
Getting app info...
Creating app with these attributes...
+ name:        pcf-config-client-pet-service
  path:        M:\pcf-config-client-pet-service\target\pcf-config-client-pet-service-0.0.1-SNAPSHOT.jar
+ instances:   1
+ memory:      1G
  services:
+   my-config-server
  env:
+   JBP_CONFIG_OPEN_JDK_JRE
+   spring.application.name
  routes:
+   pcf-config-client-pet-service-appreciative-gerenuk-qe.cfapps.io

Creating app pcf-config-client-pet-service...
Mapping routes...
Binding services...
...
Waiting for app to start...
...

Check the route of the Pet Store App

Once the deployment is completed successfully, you can check the assigned route of the app using cf app command.

$ cf app pcf-config-client-pet-service

....
routes: pcf-config-client-pet-service-appreciative-gerenuk-qe.cfapps.io
....

Test

Test pet() method

$ curl -i -X GET https://pcf-config-client-pet-service-appreciative-gerenuk-qe.cfapps.io/pet

My pet is cat

Test petDetails() method

$ curl -i -X GET https://pcf-config-client-pet-service-appreciative-gerenuk-qe.cfapps.io/petDetails

Dog people know dog language

Summary

Congratulations! You just created a new spring boot rest application which utilizes Config Server for it’s configuration.

Your feedback is always appreciated. Happy coding!

Collectors in Java

2020-01-01T00:00:00-07:00

In this article let’s go through different ways of accumulating the elements of a Stream using Collectors. This can be achived by using the collect method of Stream interface. collect takes a Collector, which is an interface for reduction operations.

Let’s use cars as our data set for examples below.

Cars Inventory:

Manufacturer	Model	Year	Price
Toyota	Corolla	2013	21000.00 $
Toyota	Camry	2018	24000.00 $
Mercedes	Benz	2019	40000.00 $

Filtering a Stream
Converting a Stream to a Map
Calculating averages
Calculating sum
Calculating sum across each group
Get costliest and cheapest car
Joining the values
Stream partitioning based on a condition

Filtering a Stream

Let’s get all the Toyota cCars in our inventory using the filter method and collect the filtered cars using Collectors.toList. This is pretty basic.

List<Car> toyotaCars = cars.stream()
                .filter(car -> car.getManufacturer().equals("Toyota"))
                .collect(Collectors.toList());

[Car [manufacturer=Toyota, model=Corolla, year=2013, price=21000.0], 
Car [manufacturer=Toyota, model=Camry, year=2018, price=24000.0]]

That was simple! We used Collectors.toList which returns a collector that is sent to the collect method.

Converting a Stream to a Map

Now, let’s create a map with model as the key and Car as the value.

Map<String, Car> carMap = cars.stream()
                .collect(Collectors.toMap(Car::getModel, Function.identity()));

{
    Benz=Car [manufacturer=Mercedes, model=Benz, year=2019, price=40000.0], 
    Camry=Car [manufacturer=Toyota, model=Camry, year=2018, price=24000.0], 
    Corolla=Car [manufacturer=Toyota, model=Corolla, year=2013, price=21000.0]
}

Here, we used Collectors.toMap which takes method for creating the keys and a function for creating the values. In our example we used Car::getModel to say that we want the model as the keys.

For the value mapping, we used Function.identity. This simply returns a function that always returns its input parameter — in our case the Car — as output.

Calculating averages

Let’s look at calculating the averages.

Let’s get an average price of a car in our inventory. We can achive this using Collectors.averagingInt

Double averagePrice = cars.stream().collect(Collectors.averagingDouble(Car::getPrice));

28333.333333333332

This arithmetic collector exists for int and long as well.

Calculating sum

If we want to get total price of all of my cars in inventroy, we can use Collectors.summingDouble

Double totalPrice = cars.stream().collect(Collectors.summingDouble(Car::getPrice));

85000.0

This arithmetic collector exists for int and long as well.

Calculating sum across each group

If we want to get total price for each of the manufacturer of our cars in inventroy, we can use Collectors.groupingBy and sum the balances using Collectors.summingDouble

Map<String, Double> totalPriceByManufacturer = cars.stream()
        .collect(Collectors.groupingBy(Car::getManufacturer, Collectors.summingDouble(Car::getPrice)));

{Toyota=45000.0, Mercedes=40000.0}

Get costliest and cheapest car

Let’s say I want to get the costliest car. This can be achieved by using Collectors.maxBy which takes Comparator as it’s input argument.

Let’s define a Comparator which determines the car with highest price. We shall use this comparator in Collectors.maxBy

Comparator<Car> costliestCarComparator = (car1, car2) -> Double.compare(car1.getPrice(),
				car2.getPrice());

Optional<Car> costliestCar = cars.stream().collect(Collectors.maxBy(costliestCarComparator));

Car [manufacturer=Mercedes, model=Benz, year=2019, price=40000.0]

Similarly, cheapest car can be found by using Collectors.minBy

Optional<Car> cheapestCar = cars.stream().collect(Collectors.minBy(costliestCarComparator));

Car [manufacturer=Toyota, model=Corolla, year=2013, price=21000.0]

Joining the values

Let’s say I want a comma seperated models of all the cars. This can be achieved by Collectors.joining

String models = accounts.stream().map(Account::getAccountNumber).collect(Collectors.joining(", "));

Corolla, Camry, Benz

Stream partitioning based on a condition

Let’s say I want list of cars based on Manufacturer.

This can be achieved by combination of Predicate and Collectors.partitioningBy methods.

Predicate<Car> manufacturerPredicate = car -> car.getManufacturer().equals("Toyota");

Map<Boolean, List<Car>> mapyByManufacturer = cars.stream()
                .collect(Collectors.partitioningBy(manufacturerPredicate));

{
    false=[
           Car [manufacturer=Mercedes, model=Benz, year=2019, price=40000.0]
          ], 
    true=[
           Car [manufacturer=Toyota, model=Corolla, year=2013, price=21000.0], 
           Car [manufacturer=Toyota, model=Camry, year=2018, price=24000.0]
         ]
}

This will result in a map with two entries.

One with key true which contains the cars that satisfied the toyota car predicate.
One with key false which contains the cars that did not satisfy the toyota car predicate.

This concludes the features of Collectors class. I hope you like this aticle.

Your feedback is always appreciated. Happy coding!

Completable Future in Java

2020-01-01T00:00:00-07:00

Java 8 introduced a lot of awesome features, one of them being CompletableFuture class.

Beside implementing the Future interface, CompletableFuture also implements the CompletionStage interface. CompletionStage promises that the computation eventually will be done.

The great thing about the CompletionStage is that it offers a vast selection of methods that let us attach callbacks which will be executed on completion.

This way we can build programs in a non-blocking fashion. Let’s take few examples to see some use cases of CompletableFuture.

The simplest asynchronous computation
Attach a callback
Chaining multiple callbacks
Parallel callbacks
Handling Exceptions using exceptionally
Handling Exceptions using whenComplete
Callback on multiple computations using thenCombine
Callback on multiple computations using runAfterBoth
Callback on either of computations using runAfterBoth

The simplest asynchronous computation

Let’s start with the basics — creating a simple asynchronous computation.

Supplier<Integer> heavyMethod = () -> {
	// Some heavy computation which eventually returns an Integer
	return 10;
};

CompletableFuture<Integer> asyncFunction = CompletableFuture.supplyAsync(heavyMethod);

/* Print the result returned by heavyMethod */
Integer result = asyncFunction.get();  
System.out.println(result);

Here, supplyAsync takes a Supplier containing the heavy code we want to execute asynchronously. Once, the execution of heavyMethod is completed, the result will be printed.

Well, we can take it further by attaching a call back method which will print the result once heavyMethod returns the result.

Attach a callback

callback executes after the asynchronous computation is done.

thenAccept is one one option to add a callback. It takes a Consumer — printer — which prints the result of the heavyMethod when it’s done.

// Heavy computation which eventually returns an Integer
Supplier<Integer> heavyMethod = () -> {
	return 10;
};

// Print the request
Consumer<Integer> printer = (x) -> {
	System.out.println(x);
};

CompletableFuture<?> asyncFunction = CompletableFuture.supplyAsync(heavyMethod)
                                                      .thenAccept(printer);

asyncFunction.get();

What if we want to pass values from one call back to another and so on…Like a chain? thenAccept won’t help here as it takes an input and returns nothing. In these cases, we can use another call back feature - thenApply. thenApply takes an input and returns an output.

thenApply takes a Function which accepts an input and returns a result.

Let’s put all these into a code to see how we can use these functionalities.

Chaining multiple callbacks

Let’s extend our earlier example by addig a new method which multiplies the input and returns the result - multiply

// Heavy computation which eventually returns an Integer
Supplier<Integer> heavyMethod = () -> {
	return 10;
};

// multiply the input and return the result
Function<Integer, Integer> multiply = (x) -> {
	return x * x;
};

// Print the request
Consumer<Integer> printer = (x) -> {
	System.out.println(x);
};

CompletableFuture<?> asyncFunction = CompletableFuture.supplyAsync(heavyMethod)
                                                      .thenApply(multiply)
                                                      .thenAccept(printer);

asyncFunction.get();

Here, the response of heavyMethod will be consumed by multiply and the response of multiply will be consumed by printer, which eventually prints the value 100.

Parallel callbacks

Let’s say, once our heavyMethod is completed, we want to add as well as multiply on the same result paralelly. This can be achieved using thenApplyAsync.

// Heavy computation which eventually returns an Integer
Supplier<Integer> heavyMethod = () -> {
	return 10;
};

// add
Function<Integer, Integer> add = (x) -> {
	return x + x;
};

// multiply
Function<Integer, Integer> multiply = (x) -> {
	return x * x;
};

// Print the request
Consumer<Integer> printer = (x) -> {
	System.out.println(x);
};

CompletableFuture<Integer> asyncFunction = CompletableFuture.supplyAsync(heavyMethod);

asyncFunction.thenApplyAsync(add).thenAccept(printer);
asyncFunction.thenApplyAsync(multiply).thenAccept(printer);

asyncFunction.get(); 

100
20

add and multiply will submitted as separate tasks to the ForkJoinPool.commonPool(). This results in both the add and multiply callbacks being executed when the preceding heavyMethod is completed.

Asynchronous version is a good option when we have multiple callbacks dependent on the same computation result.

Handling Exceptions using exceptionally

Let’s consider a scenario where heavyMethod might throw an exception. We can use exceptionally function to catch the exception and handle it gracefully. exceptionally is termed as a recovery method.

In the below example, we are returning a string value NOTHING TO PRINT in exceptionally method. NOTHING TO PRINT will be our recovery output when ever our heavyMethod fails.

// Heavy computation which eventually throws NULL POINTER EXCEPTION
Supplier<String> heavyMethod = () -> {
    return ((String) null).toUpperCase();
};

// Print the message
Consumer<String> printer = (x) -> {
    System.out.println("PRINT MSG: " + x);
};

CompletableFuture<Void> asyncFunction = CompletableFuture.supplyAsync(heavyMethod)

        .exceptionally(ex -> {
            System.err.println("heavyMethod threw an exception: " + ex.getLocalizedMessage());
            return "NOTHING TO PRINT";
        }).thenAccept(printer);

asyncFunction.get();

heavyMethod threw an exception: java.lang.NullPointerException
PRINT MSG: NOTHING TO PRINT

Handling Exceptions using whenComplete

whenComplete gives us more flexiblity to handle both exceptions and the results. In the below example, we are using whenComplete to print the exception to error console before exceptionally recoveres be sending a default message to printer.

// Heavy computation which eventually throws NULL POINTER EXCEPTION
Supplier<String> heavyMethod = () -> {
    return ((String) null).toUpperCase();
};

// Print the message
Consumer<String> printer = (x) -> {
    System.out.println("PRINT MSG: " + x);
};

CompletableFuture<Void> asyncFunction = CompletableFuture.supplyAsync(heavyMethod)

        .whenComplete((String result, Throwable ex) -> {
            if (ex != null) {
                System.err.println("heavyMethod threw an exception: " + ex.getLocalizedMessage());
            }
        }).exceptionally(ex -> {
            return "NOTHING TO PRINT";
        }).thenAccept(printer);

asyncFunction.get();

heavyMethod threw an exception: java.lang.NullPointerException
PRINT MSG: NOTHING TO PRINT

Callback on multiple computations using thenCombine

Sometimes it would be really helpful to be able to create a callback that is dependent on the result of two computations. This is where thenCombine becomes handy.

thenCombine allows us to register a BiFunction callback depending on the result of two CompletionStages.

To see how this is done, let’s in addition to finding a receiver also execute the heavy job of creating some content before sending a message.

// Heavy computation which eventually returns an Integer
Supplier<Integer> heavyMethod1 = () -> {
    return 10;
};

// Heavy computation which eventually returns an Integer
Supplier<Integer> heavyMethod2 = () -> {
    return 15;
};

// Print the request
Consumer<Integer> printer = (x) -> {
    System.out.println(x);
};

CompletableFuture<Integer> asyncFunction1 = CompletableFuture.supplyAsync(heavyMethod1);
CompletableFuture<Integer> asyncFunction2 = CompletableFuture.supplyAsync(heavyMethod2);

BiFunction<Integer, Integer, Integer> sum = (result1, result2) -> {
    return (result1 + result2);
};

CompletableFuture<Void> combinedFunction = asyncFunction1.thenCombine(asyncFunction2, sum).thenAccept(printer);

combinedFunction.get(); 

In the above example, we started two asynchronous methods — heavyMethod1 and heavyMethod2. thenCombine is used to trigger sum which takes the result of both asynchronous methods and returns the result. printer prints this result.

Callback on multiple computations using runAfterBoth

runAfterBoth is another variant of thenCombine. runAfterBoth takes a Runnable not caring about the actual values of the preceding computation — only that they both are actually complete.

// Heavy computation which eventually returns an Integer
Supplier<Integer> heavyMethod1 = () -> {
	return 10;
};

// Heavy computation which eventually returns an Integer
Supplier<Integer> heavyMethod2 = () -> {
	return 15;
};

CompletableFuture<Integer> asyncFunction1 = CompletableFuture.supplyAsync(heavyMethod1);
CompletableFuture<Integer> asyncFunction2 = CompletableFuture.supplyAsync(heavyMethod2);

Runnable sum = () -> {
	System.out.println("Heavy Load methods completed successfully");
};

CompletableFuture<Void> combinedFunction = asyncFunction1.runAfterBoth(asyncFunction2, sum);

combinedFunction.get();

Heavy Load methods completed successfully

Callback on either of computations using runAfterBoth

Let’s say we have two sources of finding car details either through carfax or autocheck. We can ask both and take the result from who ever returns first.

This can be achieved by acceptEither as seen below. the consumer carDetails will be executed when either carfax or autocheck returns the result.

// Heavy computation which eventually returns an car details
Supplier<String> carfax = () -> {
    return "2013 Toyota Corolla";
};

// Heavy computation which eventually returns an car details
Supplier<String> autocheck = () -> {
    return "2013 Toyota Corolla";
};

CompletableFuture<String> carfaxResult = CompletableFuture.supplyAsync(carfax);
CompletableFuture<String> autocheckResult = CompletableFuture.supplyAsync(autocheck);

Consumer<String> carDetails = (car) -> {
    System.out.println("Car details: " + car);
};

CompletableFuture<Void> either = carfaxResult.acceptEither(autocheckResult, carDetails);

either.get();

Car details: 2013 Toyota Corolla

This concludes the features of CompletableFuture class. I hope you like this aticle.

Your feedback is always appreciated. Happy coding!

Function Composition in Java

2020-01-01T00:00:00-07:00

In this post let’s look at function composition using two compose functions provided in the Function interface - compose and andThen.

Function composition results in a reuseable function which itself is a combination of other functions.

Basically, there are two ways to achieve Function composition. They are compose and andThen.

Let’s create two functions. One which sums up and other squares up itself.

Function<Integer, Integer> sum = x -> x + x;
Function<Integer, Integer> square = y -> y * y;  

Next, let’s combine them, using compose and andThen.

Integer sumAndSquareResult = sum.compose(square).apply(3); // Returns 18
Integer squareAndSumResult = sum.andThen(square).apply(3); // Returns 36

System.out.println("sum.compose(square): " + sumAndSquareResult);
System.out.println("sum.andThen(square): " + squareAndSumResult);

18
36

The difference between compose and andThen is the order in which they execute the functions.

compose executes the parameter first (square) followed by the caller (sum)
andThen executes the caller first (sum) followed by the parameter (square)

Let’s take it further and see how we can apply composition on BiFunction functional interfaces.

Let’s use cars as our data set for examples below.

Cars Inventory:

Manufacturer	Model	Year	Price
Toyota	Corolla	2013	21000.00 $
Toyota	Camry	2018	24000.00 $
Mercedes	Benz	2019	40000.00 $

Let’s start by introducing a basic function - byManufacturer that filters cars based on Manufacturer.

BiFunction<String, List<Car>, List<Car>> byManufacturer =
        (manufacturer, cars) -> cars.stream()
            .filter(car -> car.getManufacturer().equals(manufacturer))
            .collect(Collectors.toList());

byManufacturer is a BiFunction. They take two arguments.

Let’s also create few basic functions that sorts a list of cars from costliest to cheapest(affordable I must say!) price and a function that returns the top most car in a list.

Comparator<Car> carPriceComparator = 
                    (car1, car2) -> Double.compare(car2.getPrice(),car1.getPrice());

Function<List<Car>, List<Car>> sortByPrice = 
        cars -> cars.stream()
            .sorted(carPriceComparator)
            .collect(Collectors.toList());

Function<List<Car>, Optional<Car>> first = cars -> cars.stream().findFirst();

Now that we have few functions built, let’s see how we can use them to compose new functions.

The Most Expensive Car

We can find the most expensive car in our inventroy by sorting the cars by price and then taking the top one from the sorted list.

Function<List<Car>, Optional<Car>> costliest = first.compose(sortByPrice);
Optional<Car> costliestCar = costliest.apply(cars);

System.out.println(costliestCar.isPresent() ? costliestCar.get() : null);

Car [manufacturer=Mercedes, model=Benz, year=2019, price=40000.0]

The Most Expensive Car By Toyota

Let’s see how to get the most expensive car by Toyota in our cars inventroy.

In our previous example, we wrote a function to get us the most expensive car. We can use this function in composition with a filter functions which gives all the cars based on car type to achieve this..

BiFunction<String, List<Car>, Optional<Car>> highestByCarType = byManufacturer.andThen(costliest);
Optional<Car> costliestToyotaCar = highestByCarType.apply("Toyota", cars);

System.out.println(costliestToyotaCar.isPresent() ? costliestToyotaCar.get() : null);

Car [manufacturer=Toyota, model=Camry, year=2018, price=24000.0]

Above are few examples of functional interfaces and functional compositions. The use cases around composition of functions are endless and can be customised for our needs.

Your feedback is always appreciated. Happy coding!

Setup Apache Kafka on Windows

2020-01-01T00:00:00-07:00

In this guide, let’s download and install Apache Zookeeper and Apache Kafka on Windows PC. We shall also create a topic and publish/subscribe a message to this topic.

Download and configure Apache ZooKeeper
Start the ZooKeeper Server
Download and configure Apache Kafka
Start the Kafka Server
Create a topic
Validate Topic Creation
Test Kafka Producer and Consumer

Download and configure Apache ZooKeeper

Download Apache ZooKeeper

The stable version of Apache ZooKeeper can be downloaded from Apache Zookeeper Website.

For this tutorial, I am using ZooKeeper 3.5.6 version which was downloaded from here.

Once you download the apache-zookeeper-3.5.6-bin.tar.gz, unzip it and you should be able to see the folder apache-zookeeper-3.5.6-bin.

I have it under D:\apache-zookeeper-3.5.6-bin

cd D:\apache-zookeeper-3.5.6-bin
dir

Directory of D:\apache-zookeeper-3.5.6-bin

06/10/2019  09:02 PM    <DIR>          .
06/10/2019  09:02 PM    <DIR>          ..
04/09/2019  04:13 AM    <DIR>          bin
06/10/2019  09:03 PM    <DIR>          conf
05/03/2019  05:07 AM    <DIR>          docs
06/10/2019  09:02 PM    <DIR>          lib
02/15/2019  05:55 AM            11,358 LICENSE.txt
04/09/2019  04:13 AM               432 NOTICE.txt
05/03/2019  04:41 AM             1,560 README.md
04/02/2019  06:05 AM             1,347 README_packaging.txt

Configure Apache ZooKeeper

Now that we have downloaded Apache ZooKeeper, let’s configure some basic default values so that we can run the zookeeper server.

Installation comes with a sample configuration file named zoo_sample.cfg in conf directory. Rename this zoo_sample.cfg file to zoo.cfg

ZooKeeper bootstrap broker runs on port 2181 by default. You can change the port, if you want, by modifying the value of clientPort variable in zoo.cfg file.
Also, ZooKeeper Admin runs on port 8080. You can change, if you want, by adding a new configuration admin.serverPort in zoo.cfg.
Data file will be written to /tmp/zookeeper. Change this to a valid folder in your PC by modifying the value of dataDir variable in zoo.cfg file. I have updated it to dataDir=D:/tmp/zookeeperdata.
By default, transaction logs will be written to dataDir. You can change this behavior by adding a new dataLogDir config in zoo.cfgI have set it to dataLogDir=D:/tmp/zookeeper-logs.

Start the ZooKeeper Server

Let’s go to D:\apache-zookeeper-3.5.6-bin\bin folder and execute zkServer.cmd

cd D:\apache-zookeeper-3.5.6-bin\bin
zkServer.cmd

Console output

[main:JettyAdminServer@112] - Started AdminServer on address 0.0.0.0, port 2081 and command URL /commands
[main:ServerCnxnFactory@135] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
[main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 2 selector thread(s), 24 worker threads, and 64 kB direct buffers.
[main:NIOServerCnxnFactory@686] - binding to port 0.0.0.0/0.0.0.0:2080
[main:ZKDatabase@117] - zookeeper.snapshotSizeFactor = 0.33
[main:FileSnap@83] - Reading snapshot \tmp\zookeeper\version-2\snapshot.105
[main:FileTxnSnapLog@372] - Snapshotting: 0x105 to \tmp\zookeeper\version-2\snapshot.105
[main:ContainerManager@64] - Using checkIntervalMs=60000 maxPerMinute=10000

Now that the ZooKeeper is up and running, let’s go ahead with Kafka server setup.

Download and configure Apache Kafka

Download Apache Kafka

The stable version of Apache Kafka can be downloaded from Apache Kafka Website here.

For this tutorial, I am using Kafka 2.4.0 version which was downloaded using this link.

Once you download the kafka_2.12-2.4.0.tgz, unzip it and you should be able to see the folder kafka_2.12-2.4.0.

I have it under D:\kafka_2.12-2.4.0

cd D:\kafka_2.12-2.4.0
dir

Directory of D:\kafka_2.12-2.4.0

06/10/2019  09:09 PM    <DIR>          .
06/10/2019  09:09 PM    <DIR>          ..
05/13/2019  09:18 AM    <DIR>          bin
05/13/2019  09:18 AM    <DIR>          config
05/13/2019  09:18 AM    <DIR>          libs
05/13/2019  09:10 AM            32,216 LICENSE
06/11/2019  05:25 PM    <DIR>          logs
05/13/2019  09:10 AM               336 NOTICE
05/13/2019  09:18 AM    <DIR>          site-docs

Configure Apache Kafka

Now that we have downloaded Apache Kafka, let’s configure some basic default values so that we can run the Kafka server.

Kafka needs a running ZooKeeper instance. The default destination (ZooKeeper server and client port) is pre-configured in config\server.properties. Change thie value as per your zookeeper settings.
Kafka broker runs on port 9092 by default. You can change the port, if you want, by adding a configuration value for listeners variable in config\server.properties. As an example, if you like to run kafka broker on port 9999, you need to add listeners=PLAINTEXT://localhost:9999 in config\server.properties file so that my Kafka broker runs on port 9999.
Logs will be written to /tmp/kafka-logs. Change this to a valid folder in your PC by modifying the value of log.dirs variable in config\server.properties file. I have updated it to log.dirs=D:/tmp/kafka-logs.

We now have a basic valid configuration. Time to start the Kafka server

Start the Kafka Server

Let’s go to D:\kafka_2.12-2.4.0\bin\windows folder and execute kafka-server-start.bat giving the configuration file path in command line.

cd D:\kafka_2.12-2.4.0\bin\windows
kafka-server-start.bat "D:\kafka_2.12-2.4.0\config\server.properties"

Once the start up completes without any error, you are all set up with a running Kafka server listening on port 2082.

Create a topic

Kafka installation comes with utilites which can be used to create topics. Let’s utilize it to create a simple topic My-Test-Topic.

Let’s create My-Test-Topic topic using kafka-server-start.bat utility found in the folder D:\kafka_2.12-2.4.0\bin\windows.

cd D:\kafka_2.12-2.4.0\bin\windows
kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic My-Test-Topic

Validate Topic Creation

Validate that the topic is created successfully using the kafka-topics.bat utility.

cd D:\kafka_2.12-2.4.0\bin\windows
kafka-topics.bat --list --bootstrap-server localhost:9092

Output

My-Test-Topic

Test Kafka Producer and Consumer

Now that we have a kafka topic, it’s time to test the producer and consumer on this topic.

Start the producer

Kafka comes with a command line utility that will take input from a file or from standard input and send it out as messages to the Kafka cluster. By default, each line will be sent as a separate message.

Start the producer and then type a few messages into the console to send to Kafka topic.

cd D:\kafka_2.12-2.4.0\bin\windows
kafka-console-producer.bat --broker-list localhost:9092 --topic My-Test-Topic
Hello from Codeaches

Start the consumer to read from Kafka topic

Kafka also provides a command line utility which that will dump out messages to standard output. This is the Kafka consumer.

cd D:\kafka_2.12-2.4.0\bin\windows
kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic My-Test-Topic --from-beginning

Hello from Codeaches

Summary

This concludes our guide to installing ZooKeeper and Kafka on Windows PC.

Your feedback is always appreciated. Happy coding!

Load Balancing Tomcat Instances using NGINX on Windows

2020-01-01T00:00:00-07:00

In this post, let’s download NGINX software and setup NGINX as a load balancer for a sample application running on two instances of tomcat. In this scenario, NGINX will be setup to act as a reverse proxy.

Prerequisites
Download Apache NGINX
Configure NGINX with tomcat instances
Test the sample web app using NGINX port 80

Prerequisites

As a prerequisite, one more more instances of tomcat with a deployed sample app is required for this guide. If you don’t have one, please go through this link here to setup multiple instances of tomcat and deploy a sample app.

Download Apache NGINX

Let’s start off by downloading open source windows version of NGINX web server software. The latest version can be downloaded here.

For this tutorial, I have downloaded NGINX windows version (nginx/Windows-1.17.6) here. Unzip the downloaded file to your working directory. I have downloaded it to M:\nginx-tomcat-load-balancer-setup folder and my folder structure looks like the one shown below.

cd M:\nginx-tomcat-load-balancer-setup\nginx-1.17.6
dir

Output

Directory of M:\nginx-tomcat-load-balancer-setup\nginx-1.17.6

11/19/2019  07:19 AM    <DIR>          .
11/19/2019  07:19 AM    <DIR>          ..
11/19/2019  07:19 AM    <DIR>          conf
11/19/2019  07:19 AM    <DIR>          contrib
11/19/2019  07:19 AM    <DIR>          docs
11/19/2019  07:19 AM    <DIR>          html
12/21/2019  10:43 PM    <DIR>          logs
11/19/2019  06:26 AM         3,710,464 nginx.exe
12/21/2019  10:40 PM    <DIR>          temp

Start NGINX server

On command line terminal, execute the below mentioned commands to start the NGINX server.

M:\nginx-tomcat-load-balancer-setup\nginx-1.17.6
nginx

Validate the startup

Test the NGINX server in the browser using the URL http://localhost:80/ or simply http://localhost/.

NGINX server will render a welcome page as shown below.

Now we are all set to configure our NGINX server to work as a load balancer for tomcat instances.

Configure NGINX with tomcat instances

Add tomcat server details

I have 2 instances of tomcat running on port 7070 and 9090. Each of these instances also has a sample web app deployed.

Before we proceed with NGINX setup, let’s make sure these tomcat server instances are up and running by checking the sample app url for both tomcat the instances.

Test the tomcat instance 1 app in the browser using the URL http://localhost:7070/sample/.

Similarly, test the tomcat instance 2 app in the browser using the URL http://localhost:9090/sample/.

Now that the instances are up, let’s proceed with NGINX configuration setup.

Update nginx.conf file with below configuration which needs to be within the existing http {...} block as shown here.

###########################################
# Default round robin load balancing config
###########################################
upstream my_tomcat_setup {
    server localhost:7070;
    server localhost:9090;
}

Add server details

###########################################
#
###########################################
server {
    listen 80;
    listen [::]:80;
    location / {
            proxy_redirect      off;
            proxy_set_header    X-Real-IP $remote_addr;
            proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header    X-Forwarded-Proto $scheme;
            proxy_set_header    Host $host;
            proxy_pass          http://my_tomcat_setup;
    }
}

Reload NGINX Config

Let’s reload NGINX configuration for the above changes to take effect by executing the below command in the command line.

cd M:\nginx-tomcat-load-balancer-setup\nginx-1.17.6
nginx.exe -s reload

Test the sample web app using NGINX port 80

Now, test the NGINX setup by hitting the sample app URI using port 80 in the browser using the URL http://localhost:80/sample/.

Refresh the URL and the page will be rendered by instance 1 and 2 in round robin way.

Stop the NGINX server

You can use the below command to stop the NGINX service.

cd M:\nginx-tomcat-load-balancer-setup\nginx-1.17.6
nginx.exe -s stop

nginx.exe -h will give you list of all the commands which can be used with NGINX.

Summary

This concludes our guide to setting up NGINX as a reverse proxy for multiple tomcat instances.

Your feedback is always appreciated. Happy coding!

Codeaches

Setup Kubernetes Cluster on Google Cloud Platform (GCP) and connect using Cloud SDK

Table of contents

Create a Google Cloud Platform (GCP) account

Sing up for Google Cloud Platform.

Create a new project

Create a new Kubernetes cluster on GCP

Download and configure Google Cloud SDK on local PC

Connect to GCP Kubernetes cluster using Google Cloud SDK

Kubernetes commandline setup

Try out basic kubernetes commands

Summary

Integrate embedded Apache ActiveMQ 5 (Classic) JMS Broker with Spring Boot application

Table of contents

Create spring boot starter application

Configure additional project dependencies in pom.xml

Configure Embedded ActiveMQ Broker

Configure JMS Producer

Configure JMS Consumer

Test JMS producer and consumer

Summary

Create a stream on Pivotal Cloud Foundry (PCF) using Spring Cloud Data Flow (SCDF) server

Table of contents

Prerequisites

Start SCDF Shell CLI on your PC

Register the out-of-the-box http and log spring boot apps

Create the stream

Deploy the stream

Validate the deployed stream

Test httpLogStream stream using PCF CLI

Summary

Install Spring Cloud Data Flow (SCDF) server and Skipper Server on Pivotal Cloud Foundry (PCF)

Table of contents

Prerequisites

Provision services needed for Skipper server

Deploy Skipper server

Provision services needed for SCDF server

Deploy SCDF server

Setup SCDF Shell CLI on your PC

Setup Skipper Shell CLI on your PC

Summary

Create and deploy Spring Cloud Config Client on cloudfoundry (PCF)

Table of contents

Prerequisites

Create spring boot starter application

Deploy the application to PCF

Test

Summary

Collectors in Java

Cars Inventory:

Table of contents

Filtering a Stream

Converting a Stream to a Map

Calculating averages

Calculating sum

Calculating sum across each group

Get costliest and cheapest car

Joining the values

Stream partitioning based on a condition

Completable Future in Java

Table of contents

The simplest asynchronous computation

Attach a callback

Chaining multiple callbacks

Parallel callbacks

Handling Exceptions using exceptionally

Handling Exceptions using whenComplete

Callback on multiple computations using thenCombine

Callback on multiple computations using runAfterBoth

Callback on either of computations using runAfterBoth

Function Composition in Java

Cars Inventory:

The Most Expensive Car

The Most Expensive Car By Toyota

Setup Apache Kafka on Windows

Table of contents

Download and configure Apache ZooKeeper

Start the ZooKeeper Server

Download and configure Apache Kafka

Start the Kafka Server

Register the out-of-the-box `http` and `log` spring boot apps

Test `httpLogStream` stream using PCF CLI