PDA

View Full Version : LIFX bulbs



quixote
August 19th, 2016, 05:16 AM
Has anyone considered developing a plugin for LIFX LED bulbs? They have an api available for developers.

caseyp
December 28th, 2016, 01:57 AM
I'm looking into it but I'm running into problems working with hex values in Lua. The LIFX Lan protocol is binary/hex and is difficult to deal with in Lua. It doesn't appear to me that there's any built in libraries for dealing with binary to hex conversion or big endian to little endian conversion. If anyone knows anything about this I'd appreciate some direction.

Ron
December 28th, 2016, 02:01 AM
There are a bunch of hex to binary functions under math. For example math.binaryToHexString and hexStringToBinary. There is no big-to-little or vice verse built in but that should be too hard. What is the binary you get and what is the format of it. We'll work out how to extract data from it together.

caseyp
December 28th, 2016, 11:50 AM
Hi Ron, I found those functions after posting however I'm extremely confused by them. The LIFX API walks you through building a binary packet then it shows it has to be converted to little endian hex.

This is a partial example of a header built in binary

0011 0100 0000 0000

which should then convert to a hex value of (big endian)


34 00


This would then need to be converted to little endian

00 34

What I'm most confused about are the examples in the Girder manual relating to binary always seem to show text characters. For example binaryToHexString:


print( math.binaryToHexString( "Hello World" ) )
output: "48 65 6C 6C 6F 20 57 6F 72 6C 64"

I have always known binary to be 1's and 0's so I'm extremely confused by the example given. Can you shed some light on this for me Ron?

NOTE: The LIFX example shown comes from: https://lan.developer.lifx.com/docs/building-a-lifx-packet

Thanks,
Casey

Ron
December 28th, 2016, 09:57 PM
See attached, don't confuse the binary in the math functions with base2 numbers. It's meant to indicate that the string is treated as a opaque sequence of bytes without treating it as ascii. I just happened to pick a string of bytes "Hello World" that one can type, but it's used as a sequence of bytes with the first byte having value decimal 72 or or 0x48. Anyway here is a code snippet to extract a word out of a sequence of bytes. It also fixes the endianess in one go.





function getBigEndianWord( data, pos )
local v1 = string.byte(data, pos)
local v2 = string.byte(data, pos+1)
return bit.lshift(v2,8) + v1
end


--[[
the binary (base-2) number 00110100 is 54 in base-10 aka decimal
Don't confuse binary in this context with math.**binary*** functions. Binary in

that context simply means that the string can contain any values inside including

zeros. So when you get data from a Lifx it probably is already binary trying to
print that value will result in garbage.
--]]


local bin = "\52\00" -- equal to 00110100 00000000
print( string.format("%04X", getBigEndianWord(bin,1)))

caseyp
December 29th, 2016, 08:30 AM
Where does the "\52\00" value come from Ron? I ran your code and I got "0034" as expected, I'm just not sure how you came up with the value you entered. I have used an online binary to hex converter (http://www.binaryhexconverter.com/binary-to-hex-converter) and when I enter "0011 0100 0000 0000" from the API example I get "3400" which is correct according to the example except for the fact it's bid endian. So how in girder/lua can I enter that value and get 3400? I've read your last post several times trying to get my head around it. Are you saying the built in math functions will not be useful for this? I apologize for my ignorance on this, I've never had to work in binary or hex so this is all new to me.

Thanks,
Casey

caseyp
December 29th, 2016, 09:02 AM
I did a little more digging Ron and I found the following:



local dec = tonumber("0011010000000000",2)
local hex = math.decimaltohex(dec)
print(hex) -- outputs 3400


I have to admit I don't understand your function "getBigEndianWord" but I thought I would run the result through it and I get "3433" which is not what I expected to get. Remember, my goal is 00 34. Thankfully, it looks to me like this is all the binary I'll have to deal with as I believe the rest of the packet is constructed solely of hex and is just appended to the end.

The complete header hex in the example is: ?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 66 00 00 00" The question marks designate a reserved space for packet size and will be changed once the packet is completely built.

The final packet in the example is: 31 00 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 66 00 00 00 00 55 55 FF FF FF FF AC 0D 00 04 00 00

This packet turns one or all lights green depending on which ones you send it to.

Ron
December 29th, 2016, 10:40 AM
"\52\00" -- equal to 00110100 00000000

00110100 = 52 in decimal or 34 in hexadecimal
00000000 = 0 in decimal or 0 in hexadecimal
I'm just feeding the algorithm what you would get from Lifx.

tonumber("0011010000000000",2) returns a number not a byte string. Hence putting it into getBigEndianWord fails. The value returnsed by tonumber is not equal to the byte string indicated in the lifx documents "0011010000000000"

caseyp
December 29th, 2016, 11:46 AM
I threw the following code together Ron, is it not correct? I know it's probably not very elegant as I'm not very good with Lua but I'm getting the result I expect to see.


function headerBinToHex(binStr)
local dec = tonumber(binStr,2)
local hex = math.decimaltohex(dec)
local counter = 1
local temp, val1, val2, finalHex = '', '', '', ''

for i = 1, string.len(hex) do
if counter == 1 or counter == 3 then
temp = string.sub(hex,i,i)
elseif counter == 2 then
val1 = temp .. string.sub(hex,i,i)
elseif counter == 4 then
val2 = temp .. string.sub(hex,i,i)
counter = 0

if finalHex ~= '' then
finalHex = finalHex .. ' '
end

if val1 > val2 then
finalHex = finalHex .. val2 .. ' ' .. val1
else
finalHex = finalHex .. val1 .. ' ' .. val2
end
end

counter = counter + 1
end

return finalHex
end

print(headerBinToHex("0011010000000000")) -- outputs: 00 34, which is what the example shows

Ron
December 29th, 2016, 09:09 PM
The one thing I am worried about it that you are assuming that you will get the data in a base-2 string. You will most likely not. They are using that representation in the documentation for clarity (though it appears to confuse people more than anything). You will most likely get a string of bytes, for which you can use the function I gave you. So first thing is now to get a transport/connection up that actually gives you the data from the device. Then you can figure out how to process it.

caseyp
December 29th, 2016, 09:13 PM
Well I think at this point Ron I'm not real worried about getting responses from the lights as I'm mainly just wanting to tell them what to do and as far as I can tell I won't need to listen for a response to do that. I'm basically just wanting to hard code a few scenes and an all off setting for girder control. I figure if I need to get more complicated than that I best just use their app.

Ron
December 29th, 2016, 09:19 PM
OK. Let me know how it goes!

caseyp
December 29th, 2016, 09:29 PM
Ok, will do. When I do manage to get the packet built would simple transport be my best bet for sending it via udp?

Ron
December 29th, 2016, 11:00 PM
Yes or transport.udp ( see manual under scripting, transport, UDP connections)

caseyp
December 30th, 2016, 02:10 PM
Hey Ron, I feel like I'm getting pretty close but I've run into an issue. The following function, while fairly ugly seems to be converting both binary & decimal to hex like I want. The problem I'm having is where I try to sort the values I've put in a table. It works fine for numeric but won't sort the hex portions that contain letters and I'm guessing it's because it's looking at it like a string and not as hex. The problem is if I try to convert the value to a hex value it changes the value. What can I do to fix this? Also, if you have any ideas on how to clean this mess up I'm all ears. I know there's a lot of functions in Lua I'm either not aware of or don't know how to use properly. I'm fairly certain there's a better way to split a string into pairs then the way I'm doing it but I haven't yet figured it out.




function toLsbHex(str, isBinary, requiredByteLength)

local dec = 0
local chars = math.ceil(requiredByteLength*2)

-- convert the supplied value to decimal
if isBinary then
dec = tonumber(string.gsub(str, ' ', ''),2)
else
dec = tonumber(str)
end

-- convert the decimal into hex
local rawHex = string.format("%0" .. chars .."x", dec)

local values = {}

local counter = 1
local temp = ''

-- split the hex string into pairs for sorting
for i = 1, string.len(rawHex) do
if counter == 1 then
temp = string.sub(rawHex,i,i)
else
temp = temp .. string.sub(rawHex,i,i)
counter = 0
table.insert(values, temp)
temp = ''
end

counter = counter + 1
end

-- sort the table of hex values and place the lowest value first
table.sort(values, function(a,b) return a<b end) -- doesn't seem to sort hex values because they are actually strings

-- now that the table is sorted concatenate it back into a string
local retHex = string.upper(tostring(table.concat(values)))

return retHex
end


function compare(a,b)
return a[1] < b[1]
end

caseyp
December 30th, 2016, 08:28 PM
Ron, After rereading what you sent me previously and rereading their api it occurred to me the binary portion of the head should be treated as two separate parts. I kept seeing it as 0011 0100 0000 0000 and it's actually 0011 0100 0000 0000 (2 spaces in the middle). With that in mind I'm revisiting your solution but I'm having what seems to be a ridiculous problem. In trying to build the string: \52\00 I'm having difficulty with the backslashes. See below:




binHeader = string.format("%02d",tonumber(string.gsub(binHeader, ' ', ''),2))
binHeaderPt2 = string.format("%02d",tonumber(string.gsub(binHeaderPt2, ' ', ''),2))


-- PROBLEM AREA, I've tried both of these lines along with some others and can't get \52\00
binHeader = tostring('\' .. binHeader .. '\' .. binHeaderPt2) -- causes an error because the backslash is a special character
binHeader = tostring('\\' .. binHeader .. '\\' .. binHeaderPt2) -- outputs: \\52\\00 I can't seem to get rid of the double slashes


frameHeader = frameHeader .. toLsbHex(binHeader)




function toLsbHex(dec)
local formHex = string.format("%04X", getBigEndianWord(dec,1))
return formHex
end

function getBigEndianWord( data, pos )
local v1 = string.byte(data, pos)
local v2 = string.byte(data, pos+1)
return bit.lshift(v2,8) + v1
end

Ron
December 30th, 2016, 10:14 PM
Your going the wrong way :)



-- LIFX encoder -- https://lan.developer.lifx.com/docs/introduction


local function pack32( value )

local b1 = bit.band( bit.rshift( value , 24), 255 )
local b2 = bit.band( bit.rshift( value , 16), 255 )
local b3 = bit.band( bit.rshift( value , 8), 255 )
local b4 = bit.band( value, 255 )

return string.char(b4) .. string.char(b3) .. string.char(b2) .. string.char(b1)


end


local function pack16( value )

local b1 = bit.band( bit.rshift( value , 8), 255 )
local b2 = bit.band( value, 255 )

return string.char(b2) .. string.char(b1)


end




local function unpack16( data, pos )
local v1 = string.byte(data, pos)
local v2 = string.byte(data, pos+1)
return bit.lshift(v2,8) + v1
end


-- https://lan.developer.lifx.com/docs/workflow-diagrams
-- Port 56700
if not lifxConnection then


lifxConnection = transport.udp( function ( data, address, port )
print("LIFX RECEIVED", math.binaryToHexString(data),address,port)
end)

print("trying to listen for answers", lifxConnection:listen( 56700 ))

end




-- tagged = boolean
-- source = hex string of source address "0000" if we don't care.
-- target = 8 byte hex string.
-- message type -> A number for example 102 for Set Color
local function lifxHeaderPacket( tagged, source, target, messageType, payload )




if type(tagged ) ~= "boolean" then
print("tagged must be boolean")
return false
end

if type(source) ~= "string" then
print("Source must be a string.")
return false
end

source = math.hexStringToBinary( source )
if string.len(source) ~= 4 then
print("Source must be 4 bytes long")
return false
end

if type(target) ~= "string" then
print("Target must be a 8 byte string")
return false
end

target = math.hexStringToBinary( target )
if string.len(target) ~= 8 then
print("Target must be 8 bytes long")
return false
end


local len = 0

local header = math.hexStringToBinary( "0034" )
if not tagged then
header = math.hexStringToBinary( "0014" )
end


local reserved1 = math.hexStringToBinary("00 00 00 00 00 00")
local ackReqResReq = math.hexStringToBinary("00")
local sequence = math.hexStringToBinary("00")
local protocol = math.hexStringToBinary("00 00 00 00 00 00 00 00")
local messageType = pack16(messageType)
local reserved2 = math.hexStringToBinary("0000")

local packet = header .. source .. target .. reserved1 .. ackReqResReq .. sequence .. protocol .. messageType .. reserved2 .. payload


packet = pack16( string.len(packet) + 2 ) .. packet

return packet

end


-- color is decimal number
-- saturation is a number
-- brightness is a number $ffff
-- temperature (Kelvin) e.g. 3500
-- rampTime is a number
local function buildSetColor( tagged, source, target, color, saturation, brightness, temperature, rampTime )


if type(color) ~= "number" then
print("Color must be a number")
return false
end

if type(saturation) ~= "number" then
print("saturation must be a number")
return false
end

if type(brightness) ~= "number" then
print("brightness must be a number")
return false
end

if type(temperature) ~= "number" then
print("temperature must be a number")
return false
end

if type(rampTime) ~= "number" then
print("rampTime must be a number")
return false
end

local payload = math.hexStringToBinary("00") .. pack16(color) .. pack16(saturation) .. pack16(brightness) .. pack16(temperature) .. pack32(rampTime)


return lifxHeaderPacket( tagged, source, target, 102, payload )


end










-- example use for the buildSetColor function:
local p = buildSetColor(true, "00000000", "0000000000000000", 21845, 0xffff, 0xffff, 3500, 1024)
if not p then
return
end


-- This generates the same packet as
-- https://lan.developer.lifx.com/docs/building-a-lifx-packet
print("Sending:", math.binaryToHexString( p ))




lifxConnection:send(p, '255.255.255.255', 56700)


This code should set all your lights to bright green using the exact command from the documentation. The documentation is rather confusion mixing big and little endian, hexadecimal and base-10 and base-2 numbers in one document.

caseyp
December 30th, 2016, 11:15 PM
Looks pretty good Ron, there might be hope for you yet;). Now I'm disappointed because I couldn't get mine to work. I will agree that their documentation needs some work but after about the 10th time of going over it I was starting to understand it. My biggest problem is figuring out how to get lua to do what I need. Anyway, I'm making some changes to your work so that I can save it as a file and call it from a Girder script. I also added a function to call the color change in a more friendly manor (see below). The reason for this is that these numbers correspond to the values you see on the LIFX app so it makes it a lot easier to match up you settings. It's working pretty well. Now that you've got the hard part done and working I'll go through the rest of the API and add additional functions to handle other tasks. Once I get it complete I'll send it back to you and you can do whatever you'd like with it.

Thanks for the help Ron, I really appreciate it.
Casey



-- File
function setLight(hue, saturation, brightness, kelvin, rampTime)
hue = math.ceil((hue/ 360) * 65535)
saturation = math.ceil((saturation/100) * 65535)
brightness = math.ceil((brightness/100) * 65535)

local p = buildSetColor(true, "00000000", "0000000000000000", hue, saturation, brightness, kelvin, rampTime)
print("Sending:", math.binaryToHexString( p ))
lifxConnection:send(p, '255.255.255.255', 56700)
end







-- Girder Script
require('lifx/lifx')


-- hue 1-360 degrees
-- brightness and saturaion 0-100%
-- kelvin 2500 - 9000
-- ramp time (in milliseconds (1000 ms = 1 second))


setLight(0, 100, 100, 8000, 2000)

Ron
December 30th, 2016, 11:41 PM
already done, see next message.

Ron
December 31st, 2016, 12:13 AM
On second thought here it is:



require('lifx')

lifx.setLight(120,100,100,3500,1000)



Place the attached file in <Girder>\lua\lifx

caseyp
December 31st, 2016, 08:13 AM
Looks good Ron! Once I get finished adding all the additional functionality I'll post it back up here for your review.

UPDATE: Ron, I saved the file in a Girder6/lua/lifx folder as instructed and I'ts not working.

UPDATE: Never mind Ron, I just noticed you named the file init.lua and I had it named lifx.lua so that fixed it.

caseyp
December 31st, 2016, 09:40 AM
Ron, I'm using ZeroBrane Studio for my Lua editing and debugging however whenever I try to run something that requires transport.lua I seem to run into problems. So far the only way I've been able to "require" it is with the following line however I'm running into another issue shown below.




package.path = package.path .. ';/Program Files/Promixis/Girder 6/lua/simpleTransport/?.lua'






Program starting as '"C:\Users\Casey\Downloads\ZeroBraneStudio\bin\lua.e xe" -e "io.stdout:setvbuf('no')" "C:\Users\Casey\AppData\Local\Temp\.BE6F.tmp"'.
Program 'lua.exe' started in 'C:\Program Files\Promixis\Girder 6\lua' (pid: 12828).
Debugging session started in 'C:\Program Files\Promixis\Girder 6\lua\'.
Debugging session completed (traced 0 instructions).
Program completed in 1.12 seconds (pid: 12828).
...iles/Promixis/Girder 6/lua/simpleTransport/transport.lua:5: loop or previous error loading module 'transport'
stack traceback:
[C]: in function 'require'
...iles/Promixis/Girder 6/lua/simpleTransport/transport.lua:5: in main chunk
[C]: in function 'require'
lifx/init.lua:5: in main chunk




I open and try to run transport.lua in the simpleTransport folder and I get an error at line 3: local transport = require('transport')
I'm confused as to why trasport.lua would reference itself. Is it extending an existing class elsewhere that needs to be included?

Ron
December 31st, 2016, 12:00 PM
You cannot run that file outside of Girder. It requires Girder's transport module, which is a binary not a lua file.

caseyp
December 31st, 2016, 12:28 PM
OK, that makes sense but sure makes debugging more of a pain. Another issue I've run into is I'm trying to write a function to parse the responses from the lights. I'm assuming this was why you included the unpack16 function but you didn't create an example so I'm attempting to wing it. What I think unpack does is take characters from the returned hex code and convert them to a decimal. Is this correct? Here's the code I have so far...




local function parseLifxResponse(resp, address, port)
-- SAMPLE LIFX RECEIVED MESSAGE 29 00 00 54 00 00 00 00 D0 73 D5 14 A3 B2 00 00 4C 49 46 58 56 32 00 00 04 DC 29 7B 41 67 95 14 03 00 00 00 05 7C DD 00 00 IP: 192.168.1.136:56700




print("LIFX RECEIVED:", resp, " IP: " .. address .. ":".. port)

local packetSize = unpack16(resp, 1)
print("packet size: ", packetSize) -- Outputs: 12595
print(tonumber(packetSize)) -- Results in an error(attempt to call global 'tonumber' (a nil value))
end


Anyway, I may be doing this wrong but I'm puzzled as to why tonumber isn't working in this context. Is it something to do with it being a module?

Ron
December 31st, 2016, 12:36 PM
you must feed unpack the raw binary not the hex decoded string.

caseyp
December 31st, 2016, 02:11 PM
I've tried to construct a listener based on what you have in the Girder manual and I'm getting some strange results.




function callback( connection, data, address, port )
print(data,address,port)
end


-- https://lan.developer.lifx.com/docs/workflow-diagrams
-- Port 56700
if not lifxConnection then

lifxConnection = transport.udp( function ( data, address, port )
callback(c, data, address, port)
end)


print("trying to listen for answers", lifxConnection:listen( 56700 ))
end



Results:



Sat Dec 31 15:17:36 2016 trying to listen for answers true
Sat Dec 31 15:17:36 2016 Sending: 2B 00 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 75 00 00 00 00 FF FF 03 00 00 00
Sat Dec 31 15:17:36 2016 + 192.168.1.13 56700
Sat Dec 31 15:17:36 2016 ) 192.168.1.136 56700
Sat Dec 31 15:17:36 2016 ) 192.168.1.135 56700
Sat Dec 31 15:17:41 2016 $ 192.168.1.106 48914
Sat Dec 31 15:17:41 2016 ) 192.168.1.135 56700
Sat Dec 31 15:17:41 2016 ) 192.168.1.136 56700
Sat Dec 31 15:17:41 2016 ) 192.168.1.137 56700
Sat Dec 31 15:17:41 2016 ) 192.168.1.135 56700
Sat Dec 31 15:17:41 2016 ) 192.168.1.136 56700
Sat Dec 31 15:17:41 2016 ) 192.168.1.137 56700

Ron
December 31st, 2016, 03:50 PM
tonumber is not defined inside the module, You have to explicitly include it (see local print = print at the top). That data is not unexpected. Again remember now you are printing the byte string as an ASCI string. So the first line has a +, a plus is ascii 43. The next byte is a 0 (which you don't see and strings stop printing after a zero). There is more data in fact there is 43 bytes of data there. It's just not printing feed it too math.binaryToHex to see it printed. BUT do all your unpack16 calls on the original.

caseyp
December 31st, 2016, 04:13 PM
Ok, the module thing makes sense. I just thought that since it's a global thing it would be available.

As for feeding the response directly to unpack I assume I just send the data variable to it. I tried that once and I was getting an error in the unpack function saying it was nil.

Ron
December 31st, 2016, 04:47 PM
correct you should feed it the data variable directly. It's definitely not nil, you printed it in the example above.

caseyp
December 31st, 2016, 04:50 PM
Ok Ron, I'll look into it later. I'm focusing on football now. Have a happy New Year. Thanks for the help, I'm learning quite a bit.

Ron
December 31st, 2016, 04:53 PM
Cheers, Happy New Year!

caseyp
January 2nd, 2017, 03:41 AM
Ron, I'm playing around with unpack16 and I thought at first I had it figured out but then I started getting some results I don't understand...




LIFX RESP: 29 00 00 54 CB 17 C6 09 D0 73 D5 14 A3 B2 00 00 4C 49 46 58 56 32 00 28 58 28 F8 CD FB E5 95 14 03 00 00 00 01 7C DD 00 00

print("val: ", unpack16(data, 1)) --41 (decimal of 29)
print("val: ", unpack16(data, 2)) --0 (decimal of 00)
print("val: ", unpack16(data, 4)) --52052 (I don't understand where this one came from. I would expect 54 to output 84)

How can I unpack multiple bytes at once such as "00 54" which has to be converted to binary?



Another issue I'm not sure how to handle is endianess of the response message. According to the API documentation the first 2 bytes "29 00" are the packet size. Several spots in the documentation they note putting things in little endian however "29 00" doesn't look like little endian to me. Any idea why it's this way? From what I can tell they need to be reversed and then run through math.hextodecimal("0029") to equal 41. Is this correct and how would I best handle this? I thought about splitting the entire hex string into a table then I could pull the individual bytes out as needed but I don't know if this is a good method or not.

Ron
January 2nd, 2017, 10:18 PM
print("val: ", unpack16(data, 1))

This takes the first two bytes of the data and gives you the numerical value of that. The first two bytes are 29 and 00, this is big endian. In Little endian that is 0029 (hex). 0029 hex in decimal is 41. So that is correct!

print("val: ", unpack16(data, 2))

This (mistakenly) takes the 2nd and 3rd byte. or 00 00 -> which is 0 of course. So code does what it is asked to do. You probably intended to call unpack16(data, 3) (decodes byte 3 and 4 to give you 54 00 hex or 21504. )

caseyp
January 3rd, 2017, 10:39 AM
I actually figured that out later on Ron and ended up writing an unpack8, 24, 32, 48, and 64 to handle other parts of the protocol. What I'm still puzzled by and maybe I didn't explain it well earlier is that some of their protocol appears to be little endian and other parts aren't. I'd imagine you've dealt with protocols a lot and maybe you have some idea what's going on. Anyway I am parsing most of the header now and I'm ready to start on the payload but I want to make sure I have a firm grip as to what's going on as I want this to be correct and reliable.


SAMPLE: 29 00 00 54 CB 17 C6 09 D0 73 D5 14 A3 B2 00 00 4C 49 46 58 56 32 00 28 58 28 F8 CD FB E5 95 14 03 00 00 00 01 7C DD 00 00

1) 29 00 appears to be big endian while 00 54 appears to be little endian. Am I correct about this or am i misunderstanding endianess?
2) D0 73 D5 14 A3 B2 is the MAC address, clearly not sorted as I'm betting that would change the address.
3) They only mention putting things in little endian maybe 3 or 4 times in the example, do you think that's the only portions of the protocol that should be in little endian? I thought originally all portions would be but maybe I misunderstood.

Ron
January 3rd, 2017, 10:28 PM
Yes there is some confusion about what parts are little and what parts are big. The MAC address is not reversed, you can check by taking the first 6 characters and looking up the OUI: http://aruljohn.com/mac/D073D5 . Which correctly gives lifx. Besides the weirdness at the beginning I think it's all little endian.

caseyp
January 4th, 2017, 02:12 AM
Ok, I just wanted to make sure I wasn't misunderstanding endianess. This api is fairly poorly documented. I found one reserved section that was completely left out of the example and yet it seems to function fine which leaves me to wonder if that section is really supposed to be there. Well I'll carry on and see how it goes.

caseyp
January 6th, 2017, 12:28 PM
Hey Ron, I could use your help again. I'm making good progress on the LIFX script however I'm having trouble parsing what I believe are time stamp fields called updated_at. First off, they are 64bit integers which from my brief research (see here (https://stackoverflow.com/questions/3104722/does-lua-make-use-of-64-bit-integers)) appears to be problematic for Lua. Here is what I'm getting, any thoughts?

UPDATE: I'm not sure my little endian sorting is working properly. I guess I still don't understand how to deal with it. What I'm doing is loading each of the hex bytes into a table, then sorting the table, then outputting it in whatever form I need, either hex string with no spaces, hex string with spaces, or table of hex values. Maybe it's sorting it as a string and not as a hex value.





Raw Hex String: C0 93 8C DF 96 7B 96 14
Little Endian, no spacing: 147b8c939696c0df
Online conversion to decimal: 1475927868408512735
Lua tonumber result: 4294967295 --appears to be some sort of maximum value in lua


Raw Hex String: 00 76 06 17 F6 7A 96 14
Little Endian, no spacing: 00061417767a96f6
Online conversion to decimal: 1710940864812790 --came out a smaller number than above yet this timestamp came less then a second later
Lua tonumber result: 4294967295 --same as above





My unpacking, sorting code...



local function unpackData(data, pos, bits, endian, output, removeZeros)
local bytes = bits/8
local tb = {}


for i = 1, bytes do
local str = string.byte(data, pos + (i - 1))
if str == "00" and removeZeros then
-- skip inserting the value
else
table.insert(tb, str)
end
end


if endian == "little" then
table.sort(tb)
elseif endian == "big" then
table.sort(tb, function(a, b) return a > b end)
end


local retStr = ""
for i=1, #tb do
tb[i] = string.format('%02x',tb[i])
retStr = retStr .. tb[i]


if i ~= #tb then
retStr = retStr .. " "
end
end

if output == "hexNoSpace" then -- will output a string of hex values sorted as requested with no spacing
retStr = string.gsub(retStr, " ","")
return retStr
elseif output == "hexTable" then -- will output a table of hex values sorted as requested
return tb
else -- will output a string of hex values sorted as requested with spacing
return retStr
end
end

Ron
January 7th, 2017, 12:12 PM
Yeah that doesn't look entirely right. I don't think sort is the right approach there. Did your unpack8,16,32 and 64 functions not work?

Also yes Lua doesn't do 64 bit integers. It uses doubles if I'm not mistaken. To deal with that only use part of the timestamp. Where is the documentation on that one?

caseyp
January 7th, 2017, 03:34 PM
This is just a modified version of all the unpack functions that can do any size. The results coming out are pretty much the same.


I found that Lua 5.3 can do 64bit integers but that doesn't do me any good since girder 6 uses 5.1.

The updated_at field appears in several responses however the first and most detailed explanation of it is here... https://lan.developer.lifx.com/docs/groups-and-locations

Ron
January 8th, 2017, 01:27 PM
Unfortunately update_at isn't documented so I can't tell, but I'm guessing it milliseconds since epoch. Do you need to decode this information? If so you could probably drop some accuracy before converting it to decimal.

caseyp
January 8th, 2017, 04:38 PM
I think you're right, I believe it is ms since epoch. To be honest, I don't know if I need it or not. I guess I just had it in my mind that I would decode everything but maybe I don't need to worry about it. I'm not sure how I'd truncate it but for now I'll just skip over it and come back to it if needed.

caseyp
January 9th, 2017, 04:11 AM
Ron, in a previous post you provided the following code and said it took care of the endianess. How does it take care of the endianess? Is it simply reversing the bytes without any regard to their value or does it look at the value and order them from least to greatest? If it's just reversing then how could it work on longer integers?


function getLittleEndianWord( data, pos )
local v1 = string.byte(data, pos)
local v2 = string.byte(data, pos+1)
return bit.lshift(v2,8) + v1
end



I've been playing around with it and feeding in the following and it works great but what happens when I have to feed in a longer value?
(actually I'm feeding in the raw binary packet but I'm showing the hex equivalent for illustration purposes)



29 00 00 54 00 00 00 00 D0 73 D5 14 A3 B2 00 00 4C 49 46 58 56 32 00 00 78 38 9C A0 96 0B 98 14 03 00 00 00 01 7C DD 00 00

The last 4 bytes are the port number: 7C DD 00 00

print( tonumber(string.format("%04X", getBigEndianWord(data,38)), 16)) -- prints 56700 (Correct)



As I've found, only the numeric portions of the packet are in little endian. From what I understand the rest is in the given order. What I don't understand is if it's already little endian and I'm using a windows computer which stores values in little endian then why is it out of order at all? I just don't really understand endianess and what bytes get reversed and how and it's driving me nuts.

caseyp
January 9th, 2017, 04:32 AM
I just read this Wikipedia article (https://en.m.wikipedia.org/wiki/Endianness) and I think I've got a better understanding of endianess. It looks to me like it's about numerical position and not value as I thought. So in the number 123 one is the greatest significance because it's 100 while I was thinking of it as 1. So 123 in little endian would be 321. Is that correct? In my previous thinking 513 would be reordered to 135 which I think is wrong. It would be 315, 3 being the least significant and 5 the most.

Ron
January 9th, 2017, 10:42 AM
Correct. Endianess is reversing the bytes, not sorting them! (Strictly your examples are not correct but I think you got the idea).

To manually do the endianess go via hexadecimal representation. 2 hex digits is one byte.

123 decimal is 7B in hex -> Which doesn't change! 7B if stored as 8 bit. If stored as 16 bit in little endian it becomes 7B 00, if stored as 16 bit big endian becomes 00 7B
513 decimal is 201 in hex if stored as big endian it would be 0201, if stored in little endian it would be 0102

bonus:



function getBigEndian32( data, pos )
local v1 = string.byte(data, pos)
local v2 = string.byte(data, pos+1)
local v3 = string.byte(data, pos+2)
local v4 = string.byte(data, pos+3)

return bit.lshift(v1,24) + bit.lshift(v2,16) + bit.lshift(v3,8) + v4
end

function getBigEndian16( data, pos )
local v1 = string.byte(data, pos)
local v2 = string.byte(data, pos+1)

return bit.lshift(v1,8) + v2
end


I think the code I gave before was wrong. This is the right one. Big endian stores the high bytes first.

caseyp
January 9th, 2017, 10:50 AM
The numbers I gave were for illustration purposes. I didn't mean for them to be converted to hex. My point was 1=100 and 3=3 so 1 is the more significant numeral in this case. I didn't realize until I read that article that the entire value gets flipped or stays the same. This makes a ton more sense to me as I've been trying like heck to figure out how the computer knows which order to put them back in. It's basically like this... In English we write in big endian, left to write, most significant value first, in Chinese I believe they write from right to left which would look like little endian to us but big endian to them. By George, I think I've got it! Lol