OpenWeatherMap integration

1) Introduction

Some of you may always wanted to show up the current weather at a specific location or connect weather information with your app data.

For example: In case you've got a skiing app, you only want to recommend slopes where the wether and temperature is suitable.

2) The Effort

We want to build an app where we can request weather information by city, longitude/latitude, zipcode and country. We also want to show up a 5 day forecast for a specific city. And as a small bonus, we want to list a number of weather stations nearby a location.
With the first release of our app we want it for free, because we can't estimate the number of users.
The guys from OpenWeatherMap provide some of their data for free under certain conditions.
But for our demo app it supports all features we'll need.

http://openweathermap.org/(*)


The free version got the following restrictions:

• less equal 60 requests per minute
• current weather information
• 5 days / every 3 hour forecast
• data updates in less then 2 hours
• API version support: only current version
• NO 16 days / daily forecast
• NO SSL
• NO UV-Index data
• NO bulk download

When our app works and more and more users join, we could switch to a payment model, to handle an increasing access rate.

3) The Integration

3.1.) Get your account

At first we need to sign up for an account, because we'll need an individual API-Key.

At https://home.openweathermap.org/users/sign_up (*) we can easily register.

Afterwards you'll find the API-Key in your profile information.

3.2.) Download OpenWeatherMapClient component

In the next step you're welcome to download my OpenWeatherMapClient component.

This component is platform independent (Windows 32/64, Android, iOS, MacOS).

It was built with Delphi 10.1 Berlin (should also run with older versions, where anonymous procedures and generics are supported). 
The component works with Firemonkey (FMX) and VCL applications. It uses a REST client for gathering information.
The component was designed to use the free version of OpenWeatherMap API, so only allowed methods are implemented.

Download: OpenWeatherMap.zip

3.3) Usage of OpenWeatherMap component

Inside the zip-file you can find the API client component files and our firemonkey demo app.

• OpenWeatherMap.pas: TOpenWeatherMap component
• OpenWeatherMapClient.pas: Encapsulated TRESTClient with easy API access methods
• OpenWeatherMapInterfaces.pas: interfaces representing the API structure
• OpenWeatherMapTypes.pas: Interface-Implementations
• example/fmx/OpenWeatherMapApp.dproj: demo app
• example/images/...: external weather images for displaying current weather (seperate licence)

Example: 

procedure TForm1.FormCreate(Sender: TObject);
var lWeather : ICurrentWeather;
begin
  fOpenWeatherMap := TOpenWeatherMap.Create(Self);
  fOpenWeatherMap.client.api_key := '<YOUR-API-KEY-HERE>';
  lWeather := fOpenWeatherMap.client.getCurrentWeather('London', '', '', 'metric', 'en');
end;

The component uses interfaces, so if you're not firm with those, please take a closer look at: http://docwiki.embarcadero.com/RADStudio/Seattle/de/Objekt-Interfaces (*)

The API client was encapsulated inside a TOpenWeatherMap-Component, because we'd like to use the IOpenWeatherMapClient inside an external module.

Every IWeatherObject works as a JSON Data wrapper/helper instance. So we're accessing the JSON data directly. This is quite performant when receiving the complex JSON structure. We do not interprete, instead we're only unmarshaling at first.
The data is interpreted on demand, when using a specific property (also sub-interfaces!).

A disadvantage is: when accessing the same simple property (integer, float, string or boolean) multiple times, the JSON data is requested every time.

But in most cases you're reading/writing property values only once.

4) Conclusion

As always I'd like to thank you for reading my blog. Let me know if there are some bugs, mistakes or problems with this article or component.

(*) These links are being provided as a convenience and for informational purposes only; they do not constitute an endorsement or an approval by this blog of any of the products, services or opinions of the corporation or organization or individual. This blog bears no responsibility for the accuracy, legality or content of the external site or for that of subsequent links. Contact the external site for answers to questions regarding its content.

Delphi DX Seattle: TRESTClient SSL connection

1) Introduction

Since Delphi XE8 the REST client component has changed fundamentally. The team of Embarcadero exchanged the Indy component TIdHttpClient with the platform specific native components.
In most cases we don't even recognize something has changed.
But when I recompiled my server and client application the communication was no longer given on mobile devices.

2) The Effort

The core problem I had to deal with, was the usage of an SSL connection as recommend in REST-Protocol.
Until now I used a self signed certificate created with OpenSSL.
Because Indy can ignore certificate problems easily, there was just a warning that my certificate was not trusted (f.e. in Chrome browser).

But since Delphi uses the native HTTP client components things had changed.

REMARK: I will mention here, that it is absolutely the right decision to use native clients, not because of license restrictions and missing gzip functionality, the more because of the proximity to the operating system. A simple question: Why using an external component, when platform specific components already handle everything we need and we do not have to maintain those?

Nevertheless, for SSL communication we need certificates. On Windows OS everything works still the same. But when running my client app on Android 6.0.1 (Nexus 5) the native client shows up its security features.
Because I was using a self signed certificate, Android do not trust it and blocks the communication.

Of course there are ways to ignore this restriction but they seem to me a bit dirty, f.e. instantiate an own TrustManager, which recognizes the certificate and allows access.

A solution is described at the official android documentation:
http://developer.android.com/training/articles/security-ssl.html (*)

3) The solution

At this point, I decided to say goodbye to self signed certificates!
When developing REST applications we want support a safe and user-friendly communication. The user shouldn't decide if he can trust our certificate or not.

Since lately some organizations came up by generating trusted certificates for FREE!
And now everyone can create one at those non-profit certificate authorities (CA). Here's a short list of some of those organizations.
(I take no responsibility for the content)

• https://letsencrypt.org/ (*)
• http://www.cacert.org/ (*)

Nearly every commercial certificate authority also offers free certificates with a limited period. I myself tested the Comodo trial certificate at:

• https://www.instantssl.com/ (*)

The essence was: It worked! (okey, for 90 days) In the following I will describe the way to get there.

4) Instruction

For generating a certificate at first we'll need a so called CSR, Certificate Signing Request. This personal and server-specific file is going to be delivered to the certificate authority.

4.1) Download the OpenSSL binaries at: https://indy.fulgan.com/SSL/ (*) and extract them on the server, where you're running your server-application

An official certificate got dependencies, which bind it to a specific server-host including the subdomain, f.e. myserver.com is not the same like rest.myserver.com

Also it contains the company and/or the admin information (name, locale, ...).

CAUTION: WE NEED AT LEAST OpenSSL 1.0.2!!! 
I'll explain later, why...

4.2) Generating the CSR

To generate a public Certificate Signing Request (CSR) and a private key, use the folling command:

openssl req -nodes -newkey rsa:2048 -keyout myserver.key -out server.csr

Now you will be asked for your credentials:

Country Name (2 letter code) [AU]: DE
State or Province Name (full name) [Some-State]: MyState
Locality Name (eg, city) []: MyCity
Organization Name (eg, company) [Internet Widgits Pty Ltd]: MyCompany Ltd
Organizational Unit Name (eg, section) []: IT
Common Name (eg, YOUR name) []: mysubdomain.mydomain.com

Email Address []:

A challenge password []: 

An optional company name []:

The e-mail address, a challenge password and an optional company name can be left blank.

If everything worked well, you'll now find two files: Key-File myserver.key and CSR-File server.csr

It's important to keep and backup both files!

The Key-File we'll need later in our Indy-Http-Server again.

4.3) Certificate authority registration

Now it's time to choose your CA and to let them generate the certificates for you, by submitting the created CSR-File.

4.4) Received files

In my case, I received four certificate files by Comodo.

• your_domain_com.crt
• AddTrustExternalCARoot.crt
• COMODORSAAddTrustCA.crt
• COMODORSADomainValidationSecureServerCA.crt

Normally we need to convert each to the PEM format, which the TIdHttpServer understands, by:

openssl x509 -in your_domain_com.crt -out your_domain_com.pem -outform PEM

BUT this is not the correct solution! We have to combine all of them into one single file, because there are dependencies between those certificates. And if we leave out even one, the complete authentication process will fail.

4.5) Bundle certificates

There is a reason why we're getting all these certificates by COMODO. That are so called intermediate certificates.

And of course we have to implement all of them in our Indy server.

"But what the hell?! I've only got 3 properties in my SSLOptions?!"

This is why we have to create a certificate bundle. To do that, use the following command:

type your_domain_com.crt COMODORSADomainValidationSecureServerCA.crt COMODORSAAddTrustCA.crt AddTrustExternalCARoot.crt > your_domain_com_bundle.pem

This simply concatenates a number of files into a new file. The equivalent on Linux to the "type" command is "cat".

Here we'll get to the point, why we need to use at least OpenSSL 1.0.2. Because only since that version the so called chained files are supported.
Otherwise we'll get an exception by Indy on loading our bundled certificates:

error:00000000:lib(0):func(0):reason(0)

We concatenate our certificates into your_domain_com_bundle.pem file. The extension PEM is important. Indy servers only understand this file format.

4.6) Linking with Indy-Server

Afterwards we'll implement three files in our TIdHttpServer:

• Root-Certificate = AddTrustExternalCARoot.pem
• Key-File = myserver.key (which we've generated parallel to the CSR-file)
• Base-Certificate = your_domain_com_bundle.pem (bundled certificates)

lServerIOHandler := TIdServerIOHandlerSSLOpenSSL.Create(fHttpServer);
lServerIOHandler.SSLOptions.CertFile     := 'your_domain_com_bundle.pem';
lServerIOHandler.SSLOptions.KeyFile      := 'myserver.key';
lServerIOHandler.SSLOptions.RootCertFile := 'AddTrustExternalCARoot.pem';
lServerIOHandler.SSLOptions.Mode         := TIdSSLMode.sslmServer;
lServerIOHandler.SSLOptions.SSLVersions  := [sslvSSLv2, sslvSSLv3, sslvTLSv1, sslvTLSv1_1, sslvTLSv1_2];
lServerIOHandler.SSLOptions.VerifyDepth  := 0; // 0 == ALL CERTIFICATES!
lServerIOHandler.SSLOptions.VerifyMode   := [];

fHttpServer.IOHandler := lServerIOHandler;

5.) Conclusion

A few years ago, this way shouldn't be an option for a lot of people. Not everyone is a member of a company or has the money to afford an expensive certificate.

But because of security improvements, f.e. by mobile platforms and because of non-profit certificate authorities, there is no more reason to generate and use self-signed certificates.

If you find any mistakes in this description or got an addition, feel free to leave a comment.

(*) These links are being provided as a convenience and for informational purposes only; they do not constitute an endorsement or an approval by this blog of any of the products, services or opinions of the corporation or organization or individual. This blog bears no responsibility for the accuracy, legality or content of the external site or for that of subsequent links. Contact the external site for answers to questions regarding its content.