[BOUNTY: CLAIMED] Cross Platform Gateway


#1

We are going to offer a bounty to someone that can write a cross platform gateway for bluz. The gateway should run on Linux and Mac, Windows is optional (sorry @peekay123!). It should support a theoretically unlimited number of bluz boards at one time.

The main goal is to get it running on a Raspberry Pi, either the new Pi 3 with the built in Bluetooth LE or using a Bluetooth LE USB dongle. If the latter option is chosen, then the device you used should be published so people can buy the same hardware.

The protocol for the gateway isn’t exactly published, but the source code for the gateway in the bluz firmware repository is now available as a starting point. Or I will give access to the iOS and Android app source code to whomever would like to try (I would recommend these first as they are more complete and functional at the moment).

Language is up to you. Would prefer Python or Node, but anything else is fine. There are some great libraries out there that would make this easier, so starting from those would be good. Adafruit has a nice library that takes care of the cross platform: https://github.com/adafruit/Adafruit_Python_BluefruitLE. Or there are some Node examples as well: https://github.com/sandeepmistry/noble

Deliverables need to be full source code published online along with instructions for installation and use. We will test by running it on a Pi with your specified hardware, connecting up to 8 devices and leaving them connected for more than 24 hours. The winner will be determined by us based on our internal testing to the above specs, responses to this thread and repository time stamps will be used to determine who completed the task first.

The prize will be an assortment pack of devices. You will get 3 bluz boards along with 3 battery shields and 3 Proto shields. We will also give you a Raspberry Pi 3, so you can use your shiny new gateway code on some shiny new hardware.

If you are going to attempt this task, please respond here so people know and we don’t have super duplication.

Let the gateway-ing begin!


Bluz Gateway in NodeJS
When will Gateway Shield be available?
Max number of DK's connected to a Linux/Cross Platform Gateway
[BOUNTY] Bluz gateway vs redbear duo
Bluz connect to a thier party Gateway
#2

I would like to take a jab at this, most likely in Node but could I have a look at the IOS and Android app source code to start.


#3

Absolutely! Send me over your GitHub username and I will add you. The iOS repo is the cleanest at this point, so that may be the best place to start.


#4

Added you to both. I should hopefully have the gateway firmware done very soon as well if you want to look at that.

Sandeeps library for node looks like a fantastic starting point, that should cover a great deal of what you need to do.

Let me know if you have any questions or need any help.


#5

I think it direct Raspberry Pi 3 to Bluz communication would be super beneficial, so I’d like to add to the bounty :slight_smile:

I’ll send US$15 via paypal to anyone who

  • Meets all the criteria described above by eric and gets the approval of the Bluz team .
  • Does this in node.js.
  • Makes it work on the Raspberry Pi 3 without need for purchasing extra hardware.

Max nodes/mesh networking
#6

Hi eric,

could you elaborate what you would like the gateway software todo?

I would be willing to work on this (based on NodeJS probably) but I need more specific requirements.

Best regards
Tobias


#7

Ah, yes, the details! An important item I should have elaborated on more.

The gateway is basically just a proxy. So it acts as a BLE central and listens for devices, and connects to ones with a specific service. Once connected, data that is passed to the gateway needs to be forwarded to the cloud. There are some very basic protocol actions that can happen (such as bluz telling the gateway to open a socket, close a socket, etc) and they are determined by the data that comes in and it’s header. For the most part, the gateway simply reads data from a BLE characteristic and then forwards it to a TCP socket, byte for byte minus the header. The same is true for the downlink, the gateway reads from the socket and forwards the data to a BLE characteristic with the addition of a header.

So it is really just translating data from a BLE central to a TCP socket. A lot of the work for the BLE central has been done in the libraries I mentioned, it’s a matter of using those in combination with the sockets.

Let me know if that makes sense or you if you have any more questions


#8

Two questions. 1. Has anyone made any progress on this? and 2. Would a C.H.I.P. be an acceptable testing platform? It’s got Wifi and Bluetooth through the Realtek RTL8723BS chip, and I don’t have a Pi to test on. :smile: If I were to work on this, it probably wouldn’t get started for a week or so, and it would be with Python.


#9

The bounty is still open. I am perfectly fine implementing on a C.H.I.P., I have one and can test on it. I will test it for portability onto a Pi as well, but it should be pretty easy to carry-over if you use some of the libraries listed above.


#10

The last few weeks I have been back and forth with this challenge, and believe I have gotten to a point that just is out of my realm of understanding right now. Thus, in the hopes that maybe someone else will have a better understanding of how the data needs to be sent and receive this is the code I have so far: https://github.com/StuartMVG/blugate

Using Node.js it runs on a Mac and Raspberry Pi 3 (Please follow the Noble installment guide here: https://github.com/sandeepmistry/noble)
As for the trouble I am having, has to do with the data that is coming through from both sides. It will connect to the Bluz and Particle cloud, however I’m not sure of how the data needs to be prepared before going either way. You can see what comes through via the console log.
I have used a BLE sniffer to monitor the chat between the Bluz App and Bluz DK, to try and understand what is going on, but really I have no idea.

Currently, the Node script will find all BLE devices in the area, but only talk to Bluz Services, and I believe it will only work with one for now (I wanted to start with one connecting, to make sure it runs first).

Therefore, if anyone is able to finish the code and fix the data problem, please feel free to use what I have done thus far.
I will continue to try and work it out, but it has been sometime now and I just think it would be better to get a working application out for everyone to use, rather than dragging this out even longer.
As for the challenge, if you are able to using the code, we can split the reward or work something out later.

Best,
Stuart


#11

#12

So, I know I said I was going to do this in Python, but I started looking at the Noble documentation and thought I’d prefer the node implemenation. First NodeJS (and really, JS in general) programming I’ve done, so go easy. I wish I’d seen @stuartmvg’s work before I got as far as I did, and I hope I’m not stepping on your toes too much with this! Those initial steps of yours would have been helpful. Ah well, I muddled through.

Anyways, I’ve currently got it acting as a gateway for two DKs on a CHIP, and it’s been stable for several hours. I haven’t confirmed that it’s long term stable, or that it won’t destroy anything. I just thought I’d put this out for anyone else to test. Nothing in the code is CHIP specific, so it should probably run on the Pi fine, though I don’t have one to test. Also, don’t have a Mac, so no luck on that testing either. It seems to recover well from DK outages and internet outages, and I’m able to flash the DKs just fine. So, the code is located at: https://github.com/mumblepins/bluz-gateway

I tried to base the instructions off of a virgin CHIP image (the default Debian w/o a GUI), so hopefully it should work on other peoples stuff. Please test and let me know if it works!


#13

Fantastic! I am going to test this out on my Pi 3 and see how it goes. I have a CHIP also, but I don’t know if I have a monitor with a composite input, so I haven’t even turned the thing on yet. I will try and find a way and give that a shot also.

I will let you know how the testing goes from my end. Thanks!


#14

Nice, I’m glad you were able to get it done. I really was lost on the data side.
If I get a chance later, I’ll run it on my PI 3 and see how it runs. :slight_smile:


#15

I think I connected a TV to mine once. And then promptly replaced the GUI version with the noGUI version and have been using it headless ever since :smile:


#16

Made a minor, but useful addition. It now starts a server (port 3000 default), that returns json info with a get call:

curl  http://localhost:3000/connected-devices
{"c23d3a59eb16":{"id":"c23d3a59eb16","particle-id":"b1e111111111111111111111111","rssi":-62,"uptime":177.133},"d23d3a59eb16":{"id":"d23d3a59eb16","particle-id":"b1e222222222222222","rssi":-64,"uptime":176.473}}

IDs changed to protect the innocent

I’d be interested if the Pi does better on rssi values, as I have my DKs about a foot away from the CHIP, and those values seem kinda low for that. I have a feeling the antenna on it isn’t the best.


#17

Sorry for the delay, school vacation week here in NH and so things are a little hectic.

I just tried this on a Pi 3, it works!

A few things I noticed or have a question on:

  • How many devices should be able to connect at once? Is there any hard limit?
  • When I first had it running, it didn’t seem to clear out connections. So I disconnected one device then tried to connect another and it never found the second device. I restarted it and now it seems to be working ok, but I wonder if it maybe hung somewhere.
  • It is really fast! I am not sure how many packets per connection interval this can handle, but it must be 6 or 7. OTA updates are really quick, faster than an iPhone it looks like.
  • The range looks really good with the Pi. I have a DK in my mailbox (https://www.hackster.io/eely22/my-mailbox-can-text-489fb7) and it connected to it from my basement, which is easily 30 yards away (and, ya know, it’s a basement, so underground).

I am going to run some longevity tests, but I imagine it will work just fine. Sounds like this bounty is claimed!


#18
  • How many devices should be able to connect at once? Is there any hard limit?

  • As near as I can tell, it’s totally dependent on the adapter in Linux (and Windows if that gets to be working. Don’t have a BL adapter to try it with) and the CSR based adapters have a 5 connection limit. . According to the noble docs, OS X has a hard 6 connection limit. As for others, I’m thinking that’s going to be trial and error.

  • When I first had it running, it didn’t seem to clear out connections. So I disconnected one device then tried to connect another and it never found the second device. I restarted it and now it seems to be working ok, but I wonder if it maybe hung somewhere.

  • Hmm. So it just never appeared on the bluetooth scan? I’ve seen some issues where cloud connection fails, but then typically it drops the connection and tries again. Your issue appears different. Are you running it with debug enabled? If you’re running it with an init script, I put a new one in the repository that defaults to a higher level of debugging, and it’d be nice to see the log.


#19

Correct. I just tried it again and was able to recreate it. I had 3 devices connected and I turned one off. I let it sit for a few minutes, then turned it back on. The DK is still advertising but the gateway on the Pi isn’t connecting. Sometimes it seems to work and sometimes it doesn’t. Not sure if this is specific to the Pi or not.

I will try and dig into the code a little later today.


#20

Could you try the develop channel (npm install git+https://github.com/mumblepins/bluz-gateway#develop)? I think it might solve your problem. I’m still getting an issue where upon startup of the gateway, it sometimes takes several tries to connect the bluz to the cloud (but eventually does). I’m getting the ID fine, and then the cloud is sending down info, which gets sent to the bluz, which then disconnects. I don’t know much about the internals of the communications protocol, so do you have any ideas as to why that happens?