Help with setting up serial device

[FearTheDentist]

I'm trying to write a plug-in for serial control of my Denon 2930 DVD player. The Lua console reports:

Time Date Source Details Payloads
02:06:16:906 2/23/2008 Denon DVD1 Communications OK

In the Lua console I'm getting some response from the player, but nothing I can make sense of so far. For example, when I open, then close the DVD tray on the player I get the following:

DenonDVDData 0 66>;;1
DenonDVDData 100000100000049
Serial: Denon DVD1 : ERROR: Incomplete response to last command
DenonDVDData 0 66>;;1410000010000004A
Serial: Denon DVD1 : ERROR: Incomplete response to last command

I created a test command to turn the player off, but the player doesnt respond and I again get the incomplete response error.

I suspect I have 2 issues:

First, I think I've got something not quite right in my serial settings in the script (I have the port set to Even parity, 9600,N,1 in the Windows device manager as well as the script).

Second, I think the DVD serial protocol should use CallbackType = serial.CB_MARKEDLENGTH1, not CallbackType = serial.CB_TERMINATED. When I try this I get a Lua error though:

...gram Files\Promixis\Girder5\/plugins/serial/init.lua:456: attempt to index field `SendQueMutex' (a nil value)
stack traceback:
...gram Files\Promixis\Girder5\/plugins/serial/init.lua:456: in function `QueInsert'
...gram Files\Promixis\Girder5\/plugins/serial/init.lua:425: in function <...gram Files\Promixis\Girder5\/plugins/serial/init.lua:419>
(tail call): ?
(tail call): ?
[string "LivingRoom.gml:\Main\Scripting"]:1: in main chunk


I'm sure there's more to setting up the "MARKEDLENGTH" callback, but have no idea how to do it. I'm pretty sure my main problem has to do with not having the Block Check Characters that the protocol calls for, but have no idea how to implement them.

Here's a link to the serial protocol:
http://usa.denon.com/DVD-3930CI_2930CI_PROTOCOL_V2.02.pdf

A copy of my script so far is attached.

Any help would be greatly appreciated!!!

[Rob H]

Okay, having looked at the protocol I'd recommend using a callback type of CB_FIXEDLENGTH and set the length to a reasonable value (larger than any data you'll actually receive). All real responses will then be triggered as incompletes so ReceiveResponse will have to cope with that, remembering to clear the incomplete bit from the code. e.g.


ReceiveResponse = function(self, data, code)
if math.band(code, serial.INCOMPLETERESPONSETIMEOUT) > 0 then
code = serial.zerobits(code, serial.INCOMPLETERESPONSETIMEOUT)
-- process the data here
end
serial.Classes.Queued.ReceiveResponse(self, data, code)
end,
Note also that your code has 'if math.band(code, serial.RXCHAR) then' which will always return true, you need to check if that is > 0 for it to be a valid test.

Your StopBits value is wrong though - that should be 0 (which means 1)

[FearTheDentist]

Thanks for the quick reply.

I changed as you suggested, unfortunately now I get no response :(

Code attached.

[Rob H]

Looks to me as though you have to add a primitive checksum to the end of the command after the ETX, so you need to get rid of the SendTerminator. I'm guessing this is why you're getting no response.

Also 'f' doesn't seem to be a valid command, but 'F' is. However, from the manual it needs to be followed by 4 NUL bytes.

[FearTheDentist]

For simplicity's sake I'll use the command for "Play" ('@') in my test script in the future. However, maybe I should have been using the hex value to transmit commands, so "40h", not "@", etc. The way I was doing it do you end up with a string of 02h f ... 03h, thus mixing ascii and hex, or does G5 automatically convert the SendCommand value to hex (I think it does)? How do I enter a Null char? Lets say the complete command is "STX @ null null null null ETX BCCH BCCL" STX and ETX are defined in the serial settings, BCCH and BCCL are TBD when I figure out how to do a checksum. How do I send "f null null null null" via SendCommand?

Having said all that I'm pretty sure Parameter Codes are only required if called for by the command code.

Beyond having heard of the term, checksums are completely new to me. I did some reading, if I understand correctly the checksum is determined by converting each character to binary, calculating the sum, then converting back to hex. The protocol defines the checksum as including all characters from the Command Char through ETX. I'm not sure it's necessary to include null chars for empty parameter codes, but probably should include them until I know otherwise. So, by this reasoning, we would calculate the checksum of the above string "STX @ null null null null ETX" as "01000000 + 00000000 + 00000000 + 00000000 + 00000011" = 01000011, which translates to "43h" or "C" as my checksum. Am I on the right track here?

I think I get the basic concepts but am very confused on the implementation. How do I write a script to send the checksum?

-edit-

I connected with a terminal emulator and got responses such as this-
66>;;1410000010000004A0 (4= Close)
66>;;111000001000000470 (1= Disc Loading)
66>;;1A1000001000000570 (A=No Disc Present)
I noted the 7th char on each line because this is the status code (item 9 on the chart), just to give you a point of reference.
After staring at this for a while I realized that this is simply the system status, and each string can be translated correctly using the chart on p14 of the protocol.

So- looks like I'm getting meaningful data here, just can't figure out how to send it.

Grr....

[Rob H]

It looks to me as though pretty much all the commands require 4 parameters even though most of them are nulls

Try something like this (and remove the Send prefix)


Command = function(self, command, param1, param2, param3, param4)
local nul = string.char(0)
local stx = string.char(2)
local etx = string.char(3)
local cmd = stx .. command .. (param1 or nul) .. (param2 or nul) .. (param3 or nul) .. (param4 or nul) .. etx
-- now calculate the checksum
local checksum = 0
for i in 1, string.len(cmd) do
checksum = checksum + string.byte(cmd, i)
end
local high = math.floor(checksum / 256)
local low = math.mod(checksum, 256)
assert(checksum == (256 * high + low)) -- just to ensure we've split the checksum correctly
cmd = cmd .. string.char(high) .. string.char(low)
return self:SendCommand(cmd)
end,

You would call this using dvd:Command('@') for example, any additional parameters would be added using e.g. dvd:Command('C', '+')

[FearTheDentist]

Wow.

Unfortunately I'm getting a Lua error when I use this.
I added your code, removed SendStartByte etc, tried it with and without SendCommand = function...

From the Lua console I sent: DenonDVD:Command ('@')

When I included the SendCommand = function... I got this error (pretty sure I was supposed to cut the function out):
...gram Files\Promixis\Girder5\/plugins/serial/init.lua:425: attempt to call method `QueInsert' (a nil value)
stack traceback:
...gram Files\Promixis\Girder5\/plugins/serial/init.lua:425: in function <...gram Files\Promixis\Girder5\/plugins/serial/init.lua:419>
(tail call): ?
(tail call): ?
[string "LivingRoom.gml:\Main\Scripting"]:1: in main chunk


When I cut it out I got this error:
... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:37: attempt to call a number value
stack traceback:
... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:37: in function `Command'
[string "Interactive"]:1: in main chunk

Interestingly, when I called my test function i got sort of a response (this is with the old SendCommand function removed):
Serial: Denon DVD : Simple Send: 40 @
Serial: Denon DVD : Simple Receive: Data Code: 16384
Serial: Denon DVD : ERROR: No response to last command
Serial: Denon DVD : Simple Receive: Data 15 . Code: 8192
Serial: Denon DVD : ERROR: Incomplete response to last command

a Data of 15 from the unit means NAK or communications error, so we're getting something here.

My updated .Lua is attached
Thanks for the help so far, I think we're close!

[Rob H]

You should be calling self:Command('@') in your test function.

Also, you've changed the device to be of type serial.Classes.Simple rather than serial.Classes.Queued which is why QueInsert is nil

[FearTheDentist]

Also, you've changed the device to be of type serial.Classes.Simple rather than serial.Classes.Queued which is why QueInsert is nil
D'oh!

With the corrections I'm still getting this error when I trigger a command however:
... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:38: attempt to call a number value
stack traceback:
... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:38: in function <... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:31>
(tail call): ?
[string "LivingRoom.gml:\Main\Scripting"]:1: in main chunk

[Rob H]

Oops, mea culpa - line 38 has the wrong form of for loop. It should be


for i = 1, string.len(cmd) do

[FearTheDentist]

Progress!

Serial: Denon DVD : Command queued: 02 40 00 00 00 00 03 00 45 .@......E at postion 1
Serial: Denon DVD : Simple Send: 02 40 00 00 00 00 03 00 45 .@......E
Serial: Denon DVD : Simple Receive: Data 15 . Code: 8192
Serial: Denon DVD : ERROR: Incomplete response to last command


I think the problem is the checksum is wrong:
02 40 00 00 00 00 03 00 45

Checksum should be Command....ETX. 45 is the checksum for STX...ETX. We need to not include STX in the checksum.

[Rob H]

ah, okay - I missed that. Either change the for loop to start from 2 rather than 1 or add the STX to the start of the command after calculating the checksum.

[FearTheDentist]

OK- we are very close now. Only problem is the checksum is being mis-applied (for lack of a better term. for example:

Serial: Denon DVD : Simple Send: 02 40 00 00 00 00 03 00 43 .@......C

should be:
Serial: Denon DVD : Simple Send: 02 40 00 00 00 00 03 34 33 .@.....43

What's happening is the checksum is being parsed (is that the correct term?) incorrectly. The High value should be the first digit in the hex (4) and the low should be the second digit in the hex (3). Instead, the High is coming back (00) and the Low coming back (43).

I tested this by manually entering the checksum eg:
cmd = cmd .. 4 .. 3 in place of cmd = cmd .. string.char(high) .. string.char(low)

and it works- DVD player responds appropriately to the "@" command. Change it to .. 4 .. 4 and it responds properly to the "1" command etc.

I've been messing with variations on string.byte and string.char, but haven't had any luck. The closest I've come was this:
local high = math.floor(checksum / 256)
local low = math.mod(checksum, 256)
local test = high .. low
assert(checksum == (256 * high + low)) -- just to ensure we've split the checksum correctly
cmd = cmd .. string.char(test)
return self:SendCommand(cmd)

which returned this:
Serial: Denon DVD : Simple Send: 02 31 00 00 00 00 03 34 .1.....4

The Checksum is correct, but we want it in hex, not ascii, so line should read 02 31 00 00 00 00 03 33 34 .1.....34

I thought I needed to change string.char to string.byte, but this returned:
Serial: Denon DVD : Simple Send: 02 31 00 00 00 00 03 34 38 .1.....48

I think I'm misunderstanding the use of string.byte. We're sooooooo close. grr....


Also, what's an easy way to tell it to ignore "incomplete response" errors?

[Rob H]

Okay, I clearly misread the manual

try this


checksum = math.mod(checksum, 256)
local low = math.mod(checksum, 16) -- low 4 bits
local high = math.floor(checksum / 16) -- high 4 bits
low = string.format('%02x', low + string.byte('0'))
high = string.format('%02x', high + string.byte('0'))
cmd = cmd .. high .. low

[FearTheDentist]

OK- it's mostly working now. The code you posted was returning chars in the wrong format eg: 33 33 33 34 instead of 33 34. All it took to correct was to change it to:

checksum = math.mod(checksum, 256)
local low = math.mod(checksum, 16) -- low 4 bits
local high = math.floor(checksum / 16) -- high 4 bits
cmd = cmd .. high .. low

This works for all commands except when the low value is >9. For example, "Root Menu" is "G"- the checksum is 47 + 3, so should return 35 30. The script isn't doing the math though, so whats being returned is a high value of 4 and a low value of 10, resulting in a concatenated checksum of "410" instead of 50 being passed to the unit.

I know this is a simple math issue but I can't seem to figure this out (unfortunately as you may have noticed my coding skills are still mostly "under develoment" :))

Also- any advice on how to get the console to ignore "incomplete response" errors?

[Rob H]

Use my original code, except for the last line. Change that to


cmd = cmd .. string.byte(high) .. string.byte(low)


Also- any advice on how to get the console to ignore "incomplete response" errors?

In your ReceiveResponse method you just need to use

code = serial.zerobits(code, serial.INCOMPLETERESPONSETIMEOUT) before you call the inherited ReceiveResponse method.

[FearTheDentist]

I had tried that, no luck. The commands sent to the unit from:

local low = math.mod(checksum, 16) -- low 4 bits
local high = math.floor(checksum / 16) -- high 4 bits
cmd = cmd .. high .. low

work exactly as they should except when low>10. We want to keep these values in the format they're returned. When we add:

low = string.format('%02x', low + string.byte('0'))
high = string.format('%02x', high + string.byte('0'))

this puts the checksum in the wrong format and an error is generated. It's as if the hex code of the hex code of the ascii number is being passed eg if the checksum is 43, the hex of this is 34 33. The script is then taking the hex of 34 33 and passing that to the unit (33 34 33 33). Clear as mud, eh?

When I code it like this:

checksum = math.mod(checksum, 256)
local low = math.mod(checksum, 16) -- low 4 bits
local high = math.floor(checksum / 16) -- high 4 bits
low = string.format('%02x', low + string.byte('0'))
high = string.format('%02x', high + string.byte('0'))
cmd = cmd .. string.byte(high) .. string.byte(low)
return self:SendCommand(cmd)

A command of "@" returns this:
Serial: Denon DVD : Simple Send: 02 40 00 00 00 00 03 34 38 34 38 .@.....4848

when it should be this:
Serial: Denon DVD : Simple Send: 02 40 00 00 00 00 03 34 33 .@.....43

Also, every checksum is coming back 4848. Not sure why (probably something I'm not seeing...my eyes get blurry after drilling teeth all day...)

What we need is:
cmd = cmd .. X where X = the simple sum of local high + local low eg: if local high = 40 and local low = 10, then X = 50 and the resulting command will be cmd = cmd .. 50. In my prior post this was instead coming back cmd = cmd .. 4 .. 10.


The good news is I only have 2 more serial plugins to write after this :D

[Rob H]

Okay, let's try another tack

Add this line before the definition of the serial device


local conversion = {[0] = '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'}

and use


cmd = cmd + conversion[high] + conversion[low]

[FearTheDentist]

No joy, get this error:

... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:44: attempt to perform arithmetic on local `cmd' (a string value)
stack traceback:
... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:44: in function <... Files\Promixis\Girder5\/plugins/serial/DenonDVD.lua:29>
(tail call): ?
[string "LivingRoom.gml:\Main\Play"]:1: in main chunk


I never would have imagined it was this complicated to perform simple arithmetic! If I'm understanding that correctly, is the latest code intended to convert the high and low values to integers, then in the cmd = cmd... line the integers are added together? If so, wouldn't the structure be something like cmd = cmd .. (conversion[high] + conversion[low])? Obviouisly not though, as I tried that and it didn't work...

I'm attaching my code so you can tell me if I'm making some silly error.


-BTW- whoever coined the term "concatenate" needs to be flogged with a rubber hose.

[FearTheDentist]

For a minute there I thought I was getting clever...
I tried this:

checksum = math.mod(checksum, 256)
local low = math.mod(checksum, 16) -- low 4 bits
local high = math.floor(checksum / 16) -- high 4 bits
high = high .. 0
cmd = cmd .. (high + low)
Sending the "G" command (checksum of 50) produces what I thought was the deisred result:

Serial: Denon DVD : Simple Send: 02 47 00 00 00 00 03 35 30 .G.....50

...except it produces the NAK error which I assume is because 50 is not actually the correct checksum.
It seems the checksum is more involved than I thought ie in this context a checksum of 47 + 3 does not = 50 but (by following the ASCII chart) rather = 4A. I just tested that by the cmd = cmd .. 4A method and now the command "G" works.

Now I understand the flaw in my reasoning but have no clue how to correct it. grr...picking this stuff up by "reverse engineering" probably isn't the recommended method.

[Rob H]

I'm an idiot!

That should have been


cmd = cmd .. conversion[high] .. conversion[low]

Sorry about that.

[FearTheDentist]

NP- that was the first thing I tried. I'm at the office now so can't test it, but IIRC, it returned an error to the effect of "can't concatenate a value of "?"" or something like that. Also, the checksum came back as 48 48 regardless of the input.

If either of these issues make sense then great, otherwise I'll chalk it up to a syntax error on my part and mess with it again this evening.

Thanks again for all the help!

[Rob H]

That should all work since high and low should never exceed 15. If they do then there's a problem somewhere.

Try it and post the Lua file, along with the output.

No idea why the returned checksum should always be 48 48 - is that from sending the command?

[FearTheDentist]

I've attached the .lua I used. If it all looks right at a glance don't worry about it and I'll review it when I get home this evening. I don't have access to my home system right now so I re-created this from earlier attachments and posts. If it looks right then I probably just had a dumb typo in my working version. Yes, the 48 48 was from sending the command eg send '@' and checksum was 48 48. Send 'G' and again it was 48 48. Again- if the code looks OK to you then it's probably an inputting error on my part.

[Rob H]

Ah, you've left in the lines


local test = high .. low
low = string.format('%02x', low + string.byte('0'))
high = string.format('%02x', high + string.byte('0'))


Delete those and try it again.

[FearTheDentist]

Well Rob, you're now officially the shiznitz. Works perfect, thanks for all the help, if I ever make it to your side of the pond I owe you a beer or 10!!! Next up is my Yamaha sound bar :D

The only unresolved concern I have is what is the purpose of the "3" in ...Error',3) eg:


if math.band (code,serial.NORESPONSETIMEOUT) then
gir.LogMessage(self.Name, 'Communcication Error',3)
end

It's only there because I copied this code from another plugin and I'm concerned if this is some kind of hardware ID or some such it may create confusion in G5.

A separate question which may be more suited for Ron- in reading through the "optimization" you guys performed on my original DenonReceivers.lua (thanks for not brutalizing my ametuerish coding abilities) I noticed a few things which I'm sure end users greatly appreciate. You added code so it could be accessed from Device Manager and you added code so feedback from the receiver could be passed to NR. Are these things I should concern myself with at this point? If so can you help me get started?

[Rob H]

If you look at the help for gir.LogMessage you'll see that the third parameter to gir.LogMessage is the icon type, in this case a green disc with a white tick.

Re: the device manager stuff, that's where things start to get a little hairy :)

There are a couple of threads around here that discuss writing Device Manager providers.

[FearTheDentist]

Re: the device manager stuff, that's where things start to get a little hairy :)

There are a couple of threads around here that discuss writing Device Manager providers.
I think i'll leave that for a rainy day then...