SMS Messaging

Greetings all,

Currently my design uses mDashNotify(Email", …) to send email alerts from my ESP32 device. That in mind, my customers would also like to option to receive an SMS from the device instead of just email.

Is there a mechanism where one could POST to the Twillio API to originate an SMS from within the mDash infra?

Thanks!
-AD

mDash handles HTTP.Request notification:

mDashNotify("HTTP.Request", "{%Q:%Q, %Q: %Q, %Q: %Q}",
   "url", "https://a.b.c/d",
   "method", "POST",
   "body", "aa=bb&cc=dd");

See https://mdash.net/docs/api/lib.md#mdashnotify-send-arbitrary-notification

1 Like

Great, thanks for the tip!

-AD

Following up on this thread

…I’m having difficulty formatting an example POST to match the format that mDashNotify is looking for.

How would I format the following curl example for use with mDashNotify?

curl 'https://api.twilio.com/2010-04-01/Accounts/ssid/Messages.json' -X POST \
--data-urlencode 'To=+18005556666' \
--data-urlencode 'From=+18005551212' \
--data-urlencode 'Body=Hello' \
-u ssid:authtoken

Could you show an my example mDashNotify() call that would be functionally equivalent to this curl example?

Thanks again!
-AD

Off top of my head - not tested.

mDashNotify("HTTP.Request", "{%Q:%Q, %Q: %Q, %Q: %Q}",
   "url", "https://SSID:AUTHTOKEN@api.twilio.com/2010-04-01/Accounts/ssid/Messages.json",
   "method", "POST",
   "body", "To=18005556666&From=+18005551212&Body=Hello");

lsm,

Thanks for the response.

Looks like you’re using username:password in the URL which is a deprecated authentication method.

Is there a way to use an equivalent of the curl username option (e.g. “–u ssid:authtoken”) via mDashNotify?

-AD

Not at the moment - current API does not allow to set custom headers. It would be easy to add an extra param with headers though.

Note that the username/pass in the URL is deprecated for browsers.
When mDash makes HTTP request, it does not pass user/pass in the URL. Instead, it correctly constructs Authorization: Basic ... header. So my example would work fine, there is nothing deprecated about it.

To clarify, mDash rewrites user/pass in the URL to the Auth header:

           http://user:pass@foo.com            http://foo.com
                                               Authorization: Basic XXXXXXX
                                              
DEVICE  ------------------------------>  mDash ---------------------------> foo.com

Thanks for the explanation on the authentication, this makes more sense.

Having said all that, we’re still struggling to get mDashNotify() to work for HTTP POST requests. We can get things to work with the ESP-IDF HTTP client and with curl.

It would appear there is either a problem with mDashNotify() or a problem with the way we’re using it. Are there any debug options to see what final HTTP message is being generated from the mDash side and the HTTP response?

We’d prefer to avoid adding the unnecessary overhead of the ESP-IDF HTTP client, but may not have a choice if we can’t get mDashNotify() to work.

Working ESP-IDF Example:

	  snprintf(url, sizeof(url), "https://%s/%s/Messages.json", g_config.alertSmsURL, g_config.alertSmsSID);

	  snprintf(message, sizeof(message), "Hello!");

	  snprintf(parameters, sizeof(parameters), "To=%s&From=%s&Body=\"%s\"",
			  g_config.alertSmsToNumber, g_config.alertSmsFromNumber, message);

	  esp_http_client_config_t config = {
			  .url = url,
		      .username = g_config.alertSmsSID,
		      .password = g_config.alertSmsAuthToken,
		      .auth_type = HTTP_AUTH_TYPE_BASIC,
			  .method = HTTP_METHOD_POST,
			  .transport_type = HTTP_TRANSPORT_OVER_TCP,
			  .timeout_ms = 5000,
			  .buffer_size = 1000,
	  };

	  esp_http_client_handle_t client = esp_http_client_init(&config);

	  esp_http_client_set_post_field(client, parameters, sizeof(parameters));

	  esp_err_t err = esp_http_client_perform(client);

Failing mDashNotify() Example

	  snprintf(url, sizeof(url), "https://%s:%s@%s/%s/Messages.json", g_config.alertSmsSID,
			  g_config.alertSmsAuthToken, g_config.alertSmsURL, g_config.alertSmsSID);

	  snprintf(message, sizeof(message), "Hello!");

	  snprintf(parameters, sizeof(parameters), "To=\"%s\"&From=\"%s\"&Body=\"%s\"",
			  g_config.alertSmsToNumber, g_config.alertSmsFromNumber, message);

	  mDashNotify("HTTP.Request", "{%Q:%Q, %Q:%Q, %Q:%Q}",
			  "url", url,
			  "method", "POST",
			  "body", parameters);

Could you show the resulting url and body please?

Here you go:

url:
https://2is3mwsKdFj54JHDeKD1gy3BzvPKGgF5tb:awg4glHKDIqmvubtSOqFN3DqB12WnmRT1B@api.twilio.com/2010-04-01/Accounts/2is3mwsKdFj54JHDeKD1gy3BzvPKGgF5tb/Messages.json

(note: the ssid/authtoken values in the url above are not valid but are the correct length)

body:
To="+15005550111"&From="+15005550006"&Body="Hello!"

We have tried with and without quotes as well as percent encoding for the values To, From, Body. None of those options have worked.

-AD

I think + chars should be url-encoded.
Try this (notice %2b instead of + in the body):

mDashNotify("HTTP.Request", "{%Q:%Q, %Q: %Q, %Q: %Q}",
   "url", "https://SSID:AUTHTOKEN@api.twilio.com/2010-04-01/Accounts/ssid/Messages.json",
   "method", "POST",
   "body", "To=%2B18005556666&From=%2B18005551212&Body=Hello");

Unfortunately no success with the %2B encoding. Calling mDashNotify() with hard-coded arguments (no printf) does not work either.

Since we’re still spinning our wheels here, it there a way to determine what final HTTP message mDash is generating upon reception of an mDashNotify() request from a device?

Start a listening netcat on some server that is publicly accessible:

$ nc -l 1234

Then, make your device to request that URL:

mDashNotify("HTTP.Request", "{%Q:%Q, %Q: %Q, %Q: %Q}",
   "url", "https://SSID:AUTHTOKEN@YOUR_WORKSTATION_IP:1234/blah",
   "method", "POST",
   "body", "To=%2B18005556666&From=%2B18005551212&Body=Hello");

Make sure that mDash can see YOUR_WORKSTATION_IP, so it must not be behind a NAT.

You can use that netcat method to see what request is generated by curl command, then compare.

OK narrowing in on this one with the help of Post Test Server V2 (great tool, BTW).

Here’s what I’ve found:

With a successful curl POST, I see the following at the destination:

Parameters

Parameter Values
Body Hello
From +18005551212
To +18005556666

Post Body

No Body.

…which is good, just what we’d expect.

However with mDashNotify() I see the following:

Parameters

No Parameters.

Post Body

To=+18005556666&From=+18005551212&Body=Hello

…which is not so good.

It seems as if mDashNotify() is not properly parsing the contents of the ‘body’ argument into the appropriate HTTP parameters.

Can you confirm from your side?

Thanks!
-AD

I think you’re confusing things.

If there is no POST body, it’s a GET request with the query string parameters.

Do as I suggested - catch the query using netcat, both curl and mDash, and share.

While I am admittedly not an expert here, the use of POST vs. GET is dictated by the Twilio API - specific details here if you’re interested.

That being said, the POST method does work fine for curl and the esp-idf http client. Here are some additional captures to assist:

curl
{
“Timestamp”: “2020-01-22T15:34:37.625211Z”,
“Method”: “POST”,
“RemoteAddr”: “99.144.118.71”,
“ID”: 417037337,
“Headers”: {
“Accept”: ["/"],
“Authorization”: [“Basic c3NpZDp0b2tlbg==”],
“Content-Length”: [“48”],
“Content-Type”: [“application/x-www-form-urlencoded”],
“Host”: [“ptsv2.com”],
“User-Agent”: [“curl/7.54.0”],
“X-Cloud-Trace-Context”: [“3f6e521141f7e520fbd07340bf45c00d/18159785483506622308”],
“X-Google-Apps-Metadata”: [“domain=gmail.com,host=ptsv2.com”]
},
“FormValues”: {
“Body”: [“Hello”],
“From”: ["+18005551212"],
“To”: ["+18005556666"]
},
“Body”: “”,
“Files”: null,
“MultipartValues”: null
}

mDash
{
“Timestamp”: “2020-01-22T15:32:49.643352Z”,
“Method”: “POST”,
“RemoteAddr”: “2a01:4f8:202:14eb::2”,
“ID”: 401185587,
“Headers”: {
“Authorization”: [“Basic c3NpZDp0b2tlbg==”],
“Content-Length”: [“44”],
“Host”: [“ptsv2.com”],
“User-Agent”: [“Go-http-client/1.1”],
“X-Cloud-Trace-Context”: [“9173f6384d147abe2857a90e45964a4a/4257635965054704488”],
“X-Google-Apps-Metadata”: [“domain=gmail.com,host=ptsv2.com”]
},
“FormValues”: {},
“Body”: “To=+18005556666\u0026From=+18005551212\u0026Body=Hello”,
“Files”: null,
“MultipartValues”: null
}

@Autodog

If you did the netcat test, which shows a raw, un-interpreted requests, I would be able to help. You did not, keeping posting data that is hard to interpret. I can’t help in this case. Please find out the exact shape of the POST body that is accepted by twilio, and form the same using mDashNotify.

Response:

Unfortunately we’re unable to run netcat as you’ve requested as we don’t have access to a publicly-accessible server that is not behind a NAT. That in mind, we’re doing the best we can to diagnose this with the limited tools we have available to us. Appreciate your understanding here.

What we can do is run curl to a local machine and have captured the following:

curl ‘http://xxx.xxx.xx.xx:1234/blah’ -X POST --data-urlencode ‘To=+18005556666’ --data-urlencode ‘From=+18005551212’ --data-urlencode ‘Body=Hello’ -u ssid:token

POST /blah HTTP/1.1

Host: xxx.xxx.xx.xx:1234

Authorization: Basic c3NpZDp0b2tlbg==

User-Agent: curl/7.54.0

Accept: */*

Content-Length: 48

Content-Type: application/x-www-form-urlencoded

To=%2B18005556666&From=%2B18005551212&Body=Hello

To summarize:

  1. The Twilio API format requirements are clear
  2. curl-generated messages meet the required format
  3. esp-idf http client-generated messages meet the required format

Exact shape of message format is now very clear - hopefully the curl -> nc local capture was helpful to that effect.

Unfortunately we are unable to find a way “form the same using mDashNotify” as you’ve requested. Believe me, we have tried!

Could really use your assistance in attempting form this message with this additional info.

-AD

It looks like curl sets Content-Type: application/x-www-form-urlencoded , but mDashNotify doesn’t.

1 Like