[BOUNTY: CLAIMED] Cross Platform Gateway

The initial part of the handshake needs to happen in a pretty specific way. When the socket to the cloud is opened, the cloud immediately sends down 40 bytes. Then bluz takes that and hands back up 256 bytes. Finally, the cloud hands back down 384 bytes. This is the major part of the handshake and must be down in that order.

That is then followed by a lot of data back and forth that is basically the cloud and DK describing their capabilities to eachother. If any pet of that fails or happens in the wrong order, the handshake will fail and the DK will disconnect.

Is the DK flashing magenta and then disconnecting? If that happens, that handshake is most likely failing.

Huh. Well, the handshake appears to go fine, so I’m not sure what happens. It’s basically sending down the 384 bytes, getting back an 18 byte message, and then it disconnects. One potential thing that I’m seeing is that as soon as I turn on notifications , I get the [03, 04] EOS back from the DK, without anything being sent first. I noticed in the iOS app a reference to something about getting the EOS first, but I have a hard time interpreting swift. Any ideas?

You can ignore that. The header of each packet contains information about what the packet is, so you should wait for the connection packet to open the socket to the cloud. You can see the header created here, it is always the first two bytes: https://github.com/bluzDK/bluzDK-firmware/blob/release-1.1.47/platform/MCU/NRF51/SPARK_Firmware_Driver/src/socket.cpp#L49

One caveat there, the header was incorrect in build 1.0.47. It is fixed in 1.1.47, but the way to work around it is to only start the connection on the first packet where there is data.

I am not sure why it would disconnect, it could be something at the BLE layer. Bluz will only disconnect if the handshake fails. Other than that, link loss or improper settings could cause disconnection, for example if the central tries to force settings outside the range of what bluz allows. Is it possible to see the reason for disconnect?

I haven’t seen any connection issues with the Pi, just that sometimes it doesn’t seem to keep scanning. But once it tries to connect, it seems to work every time.

1 Like

Alright, I ran this overnight and the devices stayed connected and were responsive this morning. We can work out the issue of re-connecting later, I am calling this one claimed!

Nice work to @mumblepins! I will get the prize package shipped out today and you should have it by the weekend. Thanks!

Also, now that the basic functionality is working, I definitely plan to add more and more to this. For example, adding a web interface on top to configure settings or see the devices connected, etc. So if you want to help participate, keep an eye out for more bounties, or just send us a message with what you would want to add. We are more than happy to help reward work with hardware. Thanks everyone!

3 Likes

It’s probably time to start a different thread with this, but oh well. It seems that once I get started on some JS, I guess I can’t stop :rolling_eyes:. I made a standalone page that can monitor the gateway server, and also get information from the cloud. It stores authentication locally, so there’s no reason it needs to be run on any specific hardware. [Check it out on Github.] (https://github.com/mumblepins/bluz-gateway-infosite)

Here’s what it kinda looks like:

2 Likes

Nice work on this as well. I was thinking about doing something similar, after you finished the gateway. I had seen this one: https://github.com/suda/particle-web-interface
and though it would be cool to build out a stand alone application with election, to monitor all of your Particle and Bluz devices. May still try using your code, if you don’t mind.

You said that it monitors both the gateway and cloud. I haven’t tried it yet, but to monitor the gateway, does it need to be running on the same system?

2 Likes

Go for it. I’m not very good at making pretty web app types of things, I mainly did it for the challenge. I didn’t know about the suda’s interface, that looks pretty.

As for getting data from the gateway, it just needs to be used on the same network (assuming that you have your gateway behind a firewall of some sort), as the gateway code exposes the basic info about the devices on http://gateway-addr:3000/connected-devices using Express. I wanted to make sure that the gateway and any sort of interface could be completely separate for failure resiliency.

It looks like it should be pretty easy to integrate what I’ve kinda done already with suda’s nifty interface.

Nice work on the gateway app - thank you! It worked first time on my Raspberry Pi 3. A good alternative to the gateway shield until the issues are fixed.

This is amazing. Many thanks for all your work.
It worked for me first time on a Pi3.
What makes this so useful is that you can adapt it to monitor the conversation between bluz dk and the particle system in all sorts of ways that I haven’t thought of yet.

PS. what I forgot is that any such monitor would need to decrypt the data and decode the messages. That could be a challenge if it involved reverse-engineering the bluz firmware. Maybe there’s some doc somewhere?

It’s also a lot cheaper than the Gateway Shield + Photon! ($45 for the former, verses $35 for the latter)

$9 for the CHIP. Though if you didn’t Kickstart it, looks like June shipping.

$14 for a Pi Zero and one of these: https://www.kickstarter.com/projects/1991736672/iot-hat-for-raspberry-pi-a-must-have-for-pi-zero

@eric … awe man. I was doing so well! Now I gotta go back to Kicstarterholics Anonymous :stuck_out_tongue:

2 Likes

Ha, I know exactly how you feel…

1 Like

Has anyone had any luck running this on a Windows system? I’ve got it installed and it sees the Bluz device but never seems to get the device connected to the internet.

I’m not able to get the debug level of information to always display even though it is set in the config.json file. I know it’s reading the settings from the file because I change the port from 3000 to 3001 and I can see it using that new port.

Thanks

I have not tried this on a windows machine, but maybe you can share a few more details on the system you are using? Version? BLE device?

Pinging @mumblepins to see if he has any thoughts…

Sure thing.

Windows 7 and 10
Latest bluz-gateway code
Pluggable USB Bluetooth adapter

Here is what I see on the screen, if I can get the debug level logging to work I’ll add what it is showing, it worked once and I could see that the gateway code would connect to the cloud and pass data but just never would connect the device to the cloud.

Master: Found peripheral with ID: f6de4849df2a and Name: Bluz DK
Master: Removing peripheral: f6de4849df2a
Master: Found peripheral with ID: f6de4849df2a and Name: Bluz DK
Master: Removing peripheral: f6de4849df2a
Master: Found peripheral with ID: f6de4849df2a and Name: Bluz DK

Thanks,

Got the debug logging to work again, had to unplug the adapter and plug it in again…should I PM the debug data? It has the buffer data being sent back and forth.

If it is super long, may be better to post it to something like a github gist, but yeah, that would be good.

I didn’t write the code for this gateway but I can help with the protocol, may be useful in debugging.

It’s not that long so here it is. I was just worried about security. I quit the logging after the bluz device started flashing magenta.

Configuration saved successfully.
Master: Found peripheral with ID: f6de4849df2a and Name: Bluz DK
Master: Peripheral a bluz
Bluz f6de4849df2a: RSSI: -59
Bluz f6de4849df2a: found service: 871e022338ff77b1ed419fb3aa142db2
Bluz f6de4849df2a: found characteristic: 871e022438ff77b1ed419fb3aa142db2
Bluz f6de4849df2a: read character found
Bluz f6de4849df2a: [ 'read', 'notify' ]
Bluz f6de4849df2a: found characteristic: 871e022538ff77b1ed419fb3aa142db2
Bluz f6de4849df2a: write character found
Bluz f6de4849df2a: [ 'read', 'writeWithoutResponse', 'write' ]
Bluz f6de4849df2a: Descriptor {
  _noble: 
   Noble {
     state: 'poweredOn',
     address: '5c:f3:70:78:15:63',
     _bindings: 
      EventEmitter {
        _state: 'poweredOn',
        _addresses: [Object],
        _addresseTypes: [Object],
        _connectable: [Object],
        _pendingConnectionUuid: null,
        _connectionQueue: [],
        _handles: [Object],
        _gatts: [Object],
        _aclStreams: [Object],
        _hci: [Object],
        _gap: [Object],
        _events: [Object],
        _eventsCount: 21,
        onSigIntBinded: [Function: bound ],
        _scanServiceUuids: [] },
     _peripherals: { f6de4849df2a: [Object] },
     _services: { f6de4849df2a: [Object] },
     _characteristics: { f6de4849df2a: [Object] },
     _descriptors: { f6de4849df2a: [Object] },
     _discoveredPeripheralUUids: [],
     _events: 
      { warning: [Object],
        stateChange: [Function],
        discover: [Function] },
     _eventsCount: 3,
     _allowDuplicates: true },
  _peripheralId: 'f6de4849df2a',
  _serviceUuid: '871e022338ff77b1ed419fb3aa142db2',
  _characteristicUuid: '871e022438ff77b1ed419fb3aa142db2',
  uuid: '2902',
  name: 'Client Characteristic Configuration',
  type: 'org.bluetooth.descriptor.gatt.client_characteristic_configuration' }
Bluz f6de4849df2a: write descriptor
Bluz f6de4849df2a: wrote descriptor
Bluz f6de4849df2a: recieved from Bluz <Buffer 00>
Bluz f6de4849df2a: notify on
Bluz f6de4849df2a: Chunk Pointer 0
Bluz f6de4849df2a: ChunkLength 2
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 02 00>
Bluz f6de4849df2a: Wrote Data and EOS to Buffer, total data length: 2
Bluz f6de4849df2a: writing: <Buffer 02 00>
Bluz f6de4849df2a: notified
Bluz f6de4849df2a: writing: <Buffer 03 04>
Bluz f6de4849df2a: recieved from Bluz <Buffer 02 b1 e2 43 c8 8f 1b b3 d3 00 00 37 a1>
Bluz f6de4849df2a: recieved from Bluz <Buffer 03 04>
Bluz f6de4849df2a: end of bluz message
Bluz f6de4849df2a: recieved from Bluz <Buffer 01 10 00 01 02 01 06 16 33 02 00 16 33 36 d0 e5 04 00 00 00>
Bluz f6de4849df2a: recieved from Bluz <Buffer 00 00 00 00 00>
Bluz f6de4849df2a: recieved from Bluz <Buffer 03 04>
Bluz f6de4849df2a: end of bluz message
Bluz f6de4849df2a: Got a full buffer, attempting to send it up, length: 23
Bluz f6de4849df2a: Not connected to cloud, trying to connect
Bluz f6de4849df2a: connecting to cloud...
Bluz f6de4849df2a: cloud client connected
Bluz f6de4849df2a: Received from spark.io: <Buffer ea ab f8 af 07 f1 38 9c d4 e8 72 c0 6c dc 5b 5e 7e 23 6b 30 a8 78 dd 81 a0 62 f9 dc 62 e1 3e 57 d4 a3 67 a3 89 57 c1 da>
Bluz f6de4849df2a: Chunk Pointer 0
Bluz f6de4849df2a: ChunkLength 40
Bluz f6de4849df2a: Sent header to Buffer
Bluz f6de4849df2a: Writing this to Buffer: <Buffer ea ab f8 af 07 f1 38 9c d4 e8 72 c0 6c dc 5b 5e 7e 23 6b 30>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer a8 78 dd 81 a0 62 f9 dc 62 e1 3e 57 d4 a3 67 a3 89 57 c1 da>
Bluz f6de4849df2a: Wrote Data and EOS to Buffer, total data length: 40
Bluz f6de4849df2a: writing: <Buffer 01 00>
Bluz f6de4849df2a: writing: <Buffer ea ab f8 af 07 f1 38 9c d4 e8 72 c0 6c dc 5b 5e 7e 23 6b 30>
Bluz f6de4849df2a: writing: <Buffer a8 78 dd 81 a0 62 f9 dc 62 e1 3e 57 d4 a3 67 a3 89 57 c1 da>
Bluz f6de4849df2a: writing: <Buffer 03 04>
Bluz f6de4849df2a: recieved from Bluz <Buffer 01 00 ab 3d 47 c2 1b 6f 29 7a c9 4c a0 27 43 80 29 e4 5c 0a>
Bluz f6de4849df2a: recieved from Bluz <Buffer c3 2e 5a 6a 73 78 6f 5f 8a 0a cc 02 5b a2 1f 31 19 09 43 68>
Bluz f6de4849df2a: recieved from Bluz <Buffer d3 04 81 0b 73 b8 16 a6 a5 29 8d 48 0f 70 4c 64 21 5c c7 75>
Bluz f6de4849df2a: recieved from Bluz <Buffer c8 31 5a 1b 49 64 f6 4a f2 48 b8 be 1f 34 b6 38 6a da 2a 2e>
Bluz f6de4849df2a: recieved from Bluz <Buffer 8d a6 77 2e 51 94 0a 3e 7c 8f 07 c1 11 58 58 db df 4e 8f 72>
Bluz f6de4849df2a: recieved from Bluz <Buffer 70 ca 1d cd 21 30 90 d8 d1 c9 3a 66 4d 81 09 ae de df d6 cd>
Bluz f6de4849df2a: recieved from Bluz <Buffer 3e 80 42 ea fb a3 70 9f b3 60 d2 52 8d 47 7f a8 7f c6 34 26>
Bluz f6de4849df2a: recieved from Bluz <Buffer e1 cb 1e 40 fd 53 ed 4e b5 1d 4b 1a d9 52 67 63 d6 ee b0 6d>
Bluz f6de4849df2a: recieved from Bluz <Buffer 1f d7 cf a8 b5 72 87 25 4d bc 20 7f 1b 1d d3 5d c6 ef 03 d8>
Bluz f6de4849df2a: recieved from Bluz <Buffer 1b 58 e3 80 c9 42 20 96 aa 78 e0 33 13 8b 98 d5 6a cc 10 7f>
Bluz f6de4849df2a: recieved from Bluz <Buffer 7e 3b fa 5f 0c 10 fd 46 14 e6 cc c2 66 2c c8 b6 ef ef a9 f1>
Bluz f6de4849df2a: recieved from Bluz <Buffer 8c 8c 63 39 ca 7e 83 11 9e 6b c8 8e 20 4d e5 29 e1 e1 ad d0>
Bluz f6de4849df2a: recieved from Bluz <Buffer 47 a4 ac b1 22 df 6d e5 04 18 6c 3f 58 ac 2b b1 cf f8>
Bluz f6de4849df2a: recieved from Bluz <Buffer 03 04>
Bluz f6de4849df2a: end of bluz message
Bluz f6de4849df2a: Got a full buffer, attempting to send it up, length: 256
Bluz f6de4849df2a: Writing to cloud:  <Buffer ab 3d 47 c2 1b 6f 29 7a c9 4c a0 27 43 80 29 e4 5c 0a c3 2e 5a 6a 73 78 6f 5f 8a 0a cc 02 5b a2 1f 31 19 09 43 68 d3 04 81 0b 73 b8 16 a6 a5 29 8d 48 ... >
Bluz f6de4849df2a: Received from spark.io: <Buffer 9b 6e ed ea 16 57 15 e9 25 a3 e2 8e 3f 49 e2 2f 6c 55 78 aa da 71 11 c3 3f 8f 3f e6 56 75 68 05 65 7e c6 69 5b b8 d2 6b f3 d3 cf ad 49 84 04 9a 5b 97 ... >
Bluz f6de4849df2a: Chunk Pointer 0
Bluz f6de4849df2a: ChunkLength 384
Bluz f6de4849df2a: Sent header to Buffer
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 9b 6e ed ea 16 57 15 e9 25 a3 e2 8e 3f 49 e2 2f 6c 55 78 aa>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer da 71 11 c3 3f 8f 3f e6 56 75 68 05 65 7e c6 69 5b b8 d2 6b>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer f3 d3 cf ad 49 84 04 9a 5b 97 39 39 4e f8 ac 9c 09 4d a4 e6>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer e6 55 80 7b 88 42 1c a9 5e d1 fe 77 4e 83 5b 75 a8 36 95 ca>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer c7 0c da cd 65 e9 56 79 9f 12 b9 e6 d2 f4 b2 83 17 ea 36 09>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 0a ef 33 c7 31 25 99 75 16 01 25 d5 0f c8 a3 fc ff 6f 1c 54>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 97 97 f3 68 44 cd 67 6e a2 11 f0 23 c9 7a a8 de 7c f0 71 86>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 08 c9 71 0f c0 27 fd 67 11 1f 09 18 bd 5e 7b ad 9f fe dc 57>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 1d d5 97 55 3f 32 ce f2 df 90 81 c7 d8 63 26 da 3b 9b ef 9a>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 6a 35 a9 7f cd df c8 44 5e e4 65 9b 62 66 2a b9 65 02 50 fd>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 7f d7 f8 a4 a0 e9 7b d7 7b c4 7b 53 25 b8 42 a4 19 d5 e1 cf>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 6b d7 2c c1 eb 4e de e0 ba fe f9 5a 47 f2 f7 5a 9f bf 0b 8c>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer f7 bc 41 8b 64 fa 1e 2f da a8 d2 4d f4 20 02 60 b4 6e 89 12>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 46 87 ed cf 29 c7 5d 19 cb 27 11 cd 3a 1a 8f 66 8f 9f ca 89>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 5a 37 7a 05 89 e9 58 88 4b b9 7e f1 e1 6a fe 0a 5e 9b 16 7c>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 4d b1 39 10 0a 9f 2c c4 01 6f 00 b0 dc 25 7e 48 bd aa 57 29>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 01 db fb 37 f2 ea 26 58 83 9a b3 af 55 3e f6 3d 6a ae 8a b6>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 35 b7 72 e1 bb cf e5 17 3f 8f 94 ef 07 91 80 51 0a 3d bb 75>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer f5 1e 44 c3 94 80 7d 63 c1 32 05 db bc 2b ec 99 8a 35 de 74>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 75 5a 99 9d>
Bluz f6de4849df2a: Wrote Data and EOS to Buffer, total data length: 384
Bluz f6de4849df2a: writing: <Buffer 01 00>
Bluz f6de4849df2a: writing: <Buffer 9b 6e ed ea 16 57 15 e9 25 a3 e2 8e 3f 49 e2 2f 6c 55 78 aa>
Bluz f6de4849df2a: writing: <Buffer da 71 11 c3 3f 8f 3f e6 56 75 68 05 65 7e c6 69 5b b8 d2 6b>
Bluz f6de4849df2a: writing: <Buffer f3 d3 cf ad 49 84 04 9a 5b 97 39 39 4e f8 ac 9c 09 4d a4 e6>
Bluz f6de4849df2a: writing: <Buffer e6 55 80 7b 88 42 1c a9 5e d1 fe 77 4e 83 5b 75 a8 36 95 ca>
Bluz f6de4849df2a: writing: <Buffer c7 0c da cd 65 e9 56 79 9f 12 b9 e6 d2 f4 b2 83 17 ea 36 09>
Bluz f6de4849df2a: writing: <Buffer 0a ef 33 c7 31 25 99 75 16 01 25 d5 0f c8 a3 fc ff 6f 1c 54>
Bluz f6de4849df2a: writing: <Buffer 97 97 f3 68 44 cd 67 6e a2 11 f0 23 c9 7a a8 de 7c f0 71 86>
Bluz f6de4849df2a: writing: <Buffer 08 c9 71 0f c0 27 fd 67 11 1f 09 18 bd 5e 7b ad 9f fe dc 57>
Bluz f6de4849df2a: writing: <Buffer 1d d5 97 55 3f 32 ce f2 df 90 81 c7 d8 63 26 da 3b 9b ef 9a>
Bluz f6de4849df2a: writing: <Buffer 6a 35 a9 7f cd df c8 44 5e e4 65 9b 62 66 2a b9 65 02 50 fd>
Bluz f6de4849df2a: writing: <Buffer 7f d7 f8 a4 a0 e9 7b d7 7b c4 7b 53 25 b8 42 a4 19 d5 e1 cf>
Bluz f6de4849df2a: writing: <Buffer 6b d7 2c c1 eb 4e de e0 ba fe f9 5a 47 f2 f7 5a 9f bf 0b 8c>
Bluz f6de4849df2a: writing: <Buffer f7 bc 41 8b 64 fa 1e 2f da a8 d2 4d f4 20 02 60 b4 6e 89 12>
Bluz f6de4849df2a: writing: <Buffer 46 87 ed cf 29 c7 5d 19 cb 27 11 cd 3a 1a 8f 66 8f 9f ca 89>
Bluz f6de4849df2a: writing: <Buffer 5a 37 7a 05 89 e9 58 88 4b b9 7e f1 e1 6a fe 0a 5e 9b 16 7c>
Bluz f6de4849df2a: writing: <Buffer 4d b1 39 10 0a 9f 2c c4 01 6f 00 b0 dc 25 7e 48 bd aa 57 29>
Bluz f6de4849df2a: writing: <Buffer 01 db fb 37 f2 ea 26 58 83 9a b3 af 55 3e f6 3d 6a ae 8a b6>
Bluz f6de4849df2a: writing: <Buffer 35 b7 72 e1 bb cf e5 17 3f 8f 94 ef 07 91 80 51 0a 3d bb 75>
Bluz f6de4849df2a: writing: <Buffer f5 1e 44 c3 94 80 7d 63 c1 32 05 db bc 2b ec 99 8a 35 de 74>
Bluz f6de4849df2a: writing: <Buffer 75 5a 99 9d>
Bluz f6de4849df2a: writing: <Buffer 03 04>
Bluz f6de4849df2a: RSSI: -59
Bluz f6de4849df2a: RSSI: -59
Bluz f6de4849df2a: RSSI: -64
Bluz f6de4849df2a: Cloud connect closed
Bluz f6de4849df2a: trying to reconnect cloud in 1 second
Bluz f6de4849df2a: cloud reconnecting...
Bluz f6de4849df2a: connecting to cloud...
Bluz f6de4849df2a: cloud client connected
Bluz f6de4849df2a: Received from spark.io: <Buffer 52 f8 73 76 ea 2b c0 7b 1f a3 af c8 f6 8b 2e 0a 15 a4 b4 3b 54 4e 89 28 92 58 d3 a9 df 95 bd 3e 40 ba e6 32 b6 6d 2e 67>
Bluz f6de4849df2a: Chunk Pointer 0
Bluz f6de4849df2a: ChunkLength 40
Bluz f6de4849df2a: Sent header to Buffer
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 52 f8 73 76 ea 2b c0 7b 1f a3 af c8 f6 8b 2e 0a 15 a4 b4 3b>
Bluz f6de4849df2a: Writing this to Buffer: <Buffer 54 4e 89 28 92 58 d3 a9 df 95 bd 3e 40 ba e6 32 b6 6d 2e 67>
Bluz f6de4849df2a: Wrote Data and EOS to Buffer, total data length: 40
Bluz f6de4849df2a: writing: <Buffer 01 00>
Bluz f6de4849df2a: writing: <Buffer 52 f8 73 76 ea 2b c0 7b 1f a3 af c8 f6 8b 2e 0a 15 a4 b4 3b>
Bluz f6de4849df2a: writing: <Buffer 54 4e 89 28 92 58 d3 a9 df 95 bd 3e 40 ba e6 32 b6 6d 2e 67>
Bluz f6de4849df2a: writing: <Buffer 03 04>
Bluz f6de4849df2a: RSSI: -59
Bluz f6de4849df2a Disconnected
Master: Removing peripheral: f6de4849df2a
Bluz f6de4849df2a: Cloud connect closed
Master: Found peripheral with ID: f6de4849df2a and Name: Bluz DK
Master:  1 peripherals left
Master: Removing peripheral: f6de4849df2a
Master:  0 peripherals left
Terminate batch job (Y/N)?