DRAFT: ZMQ messages from HALUI to GUIs

[c-morley]

This is the on going work started here:
#3528
This generalizes the idea for messages.

There are three proof of concept sims:
axis_halui_test.ini
gmoccapy_halui_test.ini
qtdragon_halui_test.ini

They will load the respective gui and a separate simulated control panel.
You can notice the jograte slider on the control panel changes the jog rate on the gui.
I would love it if someone could test these rough changes.

[zz912]

Hello,

I made basic manual tests for:
axis_halui_test.ini
gmoccapy_halui_test.ini
qtdragon_halui_test.ini

Everything works as I would expect. I was able to make it go into unexpected states by accidentally clicking on it, but I don't think this panel would be expected to be robust enough for regular CNC operation. These unexpected states were not related to the zmq technology. I would rather say that the external control would need to include more of the functions provided by the GUI, or more Panel-GUI cooperation.

I don't understand what "dialog control" is for.

I tested it in all GUIs, but it doesn't do anything.
In AXIS it just prints it in the terminal:

GUI OK command called
Write Msg called
[HAL bridge][DEBUG]  Sending ZMQ Message:STATUSREQUEST {"FUNCTION": "request_ok", "ARGS": true} (bridge.py:142)
[DEFAULT.COMMON.HAL_GLIB][DEBUG]  REQUESTED:{'FUNCTION': 'request_ok', 'ARGS': True} (hal_glib.py:410)
dialog control <hal_glib.GStat object at 0x7f2ce4cefbc0 (common+hal_glib+GStat at 0xaaf3980)> 1

[zz912]

I understand that you want to unify different GUIs with zmq technology. That's why I don't understand why you copied panel.ui to all GUIs?

I think it would be appropriate to rename panel.ui to panel_halui.ui.
The name of the main window "QTvcp-Panel-panel" is strange. I suggest renaming it to "HALUI control"

However, I have no idea what your plans are for this panel in the future.

[zz912]

/* pin for running program */
I think, this is ctrl+c ctrl+v bug.
What is different between cycle and program?

[zz912]

What is the "HALBRIDGE = hal_bridge" command in an ini file good for?

In some ini files you use it, in some you don't, and in some it is commented out.
I mean the ini files in this PR.

[zz912]

Feel free to tell me what I should focus on in testing.

[c-morley]

Thank you for testing - I will try to answer your questions.

Yes the control panel was just for basic testing, it would be nice to come up with a list of 'standard' buttons that each GUI would be expected to support. each GUI requires some code work to fully integrate.

Dialog control is for the popup dialogs such as tool change or warnings. I believe I only coded Qtdragon to respond for manual tool changes and warnings when unpausing the spindle when the spindle is auto lifted.

Panel.ui is just for testing the basic code - I have put no thought into final naming and most likely would build a gladevcp based panel for gmoccapy. I just copied and pasted for speed.

program_run starts the program thru linuxcnc's command channel (so bypasses the GUI).
cycle_start requests the GUI to start the program, allowing the GUI to do other things such as pre-checks.
cycle_pause is similar. Qtdragon uses it to start the spindle and lower the spindle (when in spindle lift mode).

I would be interested if HALUI INI MDI commands work better when they are a 'request' form HALUI for the GUI to run them. As I remember, you had some timing problems in Gmoccapy.
I will check to see if I finished the Gmoccapy INI MDI test code.

[c-morley]

I see the MDI buttons work in the gmoccapy test panel. Can you simulate the problem you had with HALUI MDI commands?

[zz912]

I had three problems:
First problem wasnt caused by MDI_commands. I had T6 in MDI_commands. I did not know, where was problem. That is why I wrote aboat problematic MDI comnands. Problem was: T6 in Gmoccapy caused running another command (G43). Now it is solved by removing automatic G43.
The second problem is, that Halui during MDI_commands switching modes. So GUI must be preapered for this situation. Like this #3441
The third problem is probably interesting for you.
I will test it in weekend. https://forum.linuxcnc.org/38-general-linuxcnc-questions/49445-python-command-wait-complete-does-not-wait-for-c-halui-commands

[c-morley]

I think both problems are improved by using this halui technique.
HALUI just asked the GUI to run the MDI (using ZMQ messages)
In this way the GUI can decide about mode switching and display changes.

I'll be interested in what you find.

[zz912]

Hi,

I need to explain more what is the goal of implementing ZMQ?
Can you draw a block diagram of how ZMQ should work?
Just draw it with a pencil on paper and take a picture.

I studied the codes of this PR today. I am even more confused.

Previously, there were halui.mdi_commands in halui. These commands were triggered by the hal signal. The halui.cc file switched modes and triggered MDI commands. That was it.

Now there is halui.gui.mdi-command-MACRO0. These commands are triggered by the hal signal as well.
These commands are in the files:
halui.cc
gmoccapy.py
bridge.py
gtk_action.py
I didn't understand why it is in so many places, nor how it works there.

[c-morley]

[c-morley]

The HALUI pin halui.gui.mdi-command-MACRO0 uses ZMQ messages to request the GUI to run the commands it's self.

The best place to make sure mode changes and widget changes that are in sync with the MDI commands is the GUI.

HALUI is probably the most common way to connect a panel to the GUI, but because it thinks it's the only UI, some things like MDI and jogging do not work so well.

HALUI and bridge are really the same program. Bridge is a python library pulled into HALUI.
Bridge looks in the INI to see what macro pins need to be made.
It also reads/sends the ZMQ messages to ask Gmoccapy to run the INI macro command
It does similar things for jog rate and axis selection.

Gmoccapy use the library qtk_action to run the INI macro command.

The key here is that Gmoccapy is the one that runs the INI macro command, so it can be programed to set appropriate widgets and modes without having to worry about what HALUI is doing.

[zz912]

Gmoccapy use the library qtk_action to run the INI macro command.

Not gtk_action.py?

[zz912]

Thank you for picture. Right now, I can only be your rubber duck until I understand the whole thing.

The best place to make sure mode changes and widget changes that are in sync with the MDI commands is the GUI.

@Sigma1912 made something similar #3504
My problems with halui.mdi_commands will be solved here: #3441
So I still don't see the benefits of this PR.
ZMQ is realtime technology, but GUI, halui, bridge are userspace component. It is wierd combination for me now.

Why is bridge just a library?
What else will it be used for?
If bridge were a replacement for halui, it would be easier to maintain.

Could you describe to me better how MACRO0 works when using Gmoccapy?

The communication between the control panel and the halui is clear to me.
For MACRO0 HAL2 and HAL3 and NML1 it is not used. Is that right?

MOTION CONTROLLER is still some kind of abstract thing for me. Which specific files are these?
Let's limit ourselves to just running MACRO0, let's not make the pictures complicated.

gtk_action.py / qtk_action.py should be in the GUI image.
What does gtk_action.py / qtk_action.py do? Does it work with NML4 and HAL3?

[c-morley]

Gmoccapy use the library qtk_action to run the INI macro command.

Not gtk_action.py?

I miss spelled gtk_action

[c-morley]

ZMQ is not realtime.
For a macro call:

1. HAL is used from control panel to HALUI
2. ZMQ is used between HALUI and GUI
3. NML is used between the GUI and the MOTION CONTROLLER (aka Linuxcnc Gcode motion controller)

gtk_action is just a library of python functions for doing GUI related chores such as sending MDI commands defined in the INI.

bridge is part of HALUI, it extends HALUI to do the new functions. It can not work without HALUI.

While there are other ways to work around these problems, I believe this is better and also it covers all the GUIs (when it;s finished) and a consistent interface for users.

To summarize:
This code extends HALUI to request the GUI to run INI defined MDI commands and sync axis selection and jograte between HALUI and the GUI.
Having the GUI run the INI MDI commands (rather then HALUI) eliminates any mode state problems and allows the GUI to decide how to react to the request.
Having Jograte and axis selection in sync allows better integration of a control panel and screen controls.
Right now GUI screen jog controls are completely separate from HALUI jog controls.
Finally, having the interface the same between all GUIs makes it easier for users to set up.

[c-morley]

Here is a better diagram (The other was slightly wrong with the HAL connections to the motion controller from HALUI

[zz912]

ZMQ is not realtime.

Is NML realtime?

[c-morley]

No.
https://linuxcnc.org/docs/devel/html/code/code-notes.html

[zz912]

Do you think that HAL between GUI and HALUI should be replaced by ZMQ in the future?
In an ideal world.

[c-morley]

Possibly but certainly not necessarily. I think the things that are very standard (say jogging, axis selection) are better done in standard way (in this case I chose HALUI) that does not require connecting HAL pins.

I would bet that non standard things would still be added using HAL pins - it's easy to do

[c-morley]

Well I said without connecting HAL pins, but I meant not connecting HAL pins from the GUI to HALUI, you still connect some HAL pins in HALUI, say for a control panel.
It just makes things standard and easier, while still allowing much customization.

[zz912]

I always wondered why we need NML or ZMQ when theoretically we could be done via HAL?

[c-morley]

Outside my cone of hard knowledge, but as I under stand it, NML is a Neutral Message Layer for networking. I believer the idea was the GUI, IO controller and motion controller could be on different computers. Certainly the GUI side could be networked at one time.
HAL uses shared memory scheme, so can not directly be networked.

[zz912]

What advantages does ZMQ have over HAL?

[c-morley]

In this case, zmq automically connects the gui to halui and any other program that registers to it. This creates consistent information in the system and a consistent connection path for doing jogging or axis selection.

[zz912]

Thank you for explaining your thoughts.

Now I can get back to testing.

My test procedure:
1 ) Run gmoccapy_halui_test.ini
2 ) Turn off E-stop
3 ) Turn on machine
4 ) Home machine

Test number 1:
5 ) In the control panel press Manual
6 ) In the control panel press 0
You will see an ugly mode switch.
(If you don't see it, you have a very powerful PC and you should add G4 P10 to gui_mdi_commands)
This ugly mode switch also happens when using the classic mdi_commands too. I created a fix for it #3441 .

Test number 2:
7 ) In the control panel press MDI
8 ) In the control panel press 0
There is no problem

Test number 3:
9 ) In the control panel press AUTO
10 ) In the control panel press 0
Result:

[zz912]

For another test:

You must modify in INI file:
MDI_COMMAND_MACRO0 = G91; G1 Z-10

You must modify in gmoccapy.py

# =========================================================
# Coolant an mist coolant button
    def on_tbtn_flood_toggled(self, widget, data=None):
            self.command.wait_complete(30)
            self.command.mode(linuxcnc.MODE_MDI)
            self.command.wait_complete(30)
            self.command.mdi("G91 G1 X10 F200")
            self.command.wait_complete(30)
            self.command.mdi("G91 G1 Y10 F200")
            self.command.wait_complete(30)           
            self.command.mode(linuxcnc.MODE_MANUAL)
            self.command.wait_complete()

My test procedure:
1 ) Run gmoccapy_halui_test.ini
2 ) Turn off E-stop
3 ) Turn on machine
4 ) Home machine
5) go to 0 position => g90 => g0 x0z0y0
6 ) In the control panel press 0 and wait
Result:

Everything is OK now.
7 ) In the control panel press 0 and immediately press cooling and wait for a few seconds.
Result:

You are not in position Z-20 , because self.command.wait_complete(30) doesn't wait for gui_mdi_commands

Old mdi_commands make errors:
https://forum.linuxcnc.org/38-general-linuxcnc-questions/49445-python-command-wait-complete-does-not-wait-for-c-halui-commands

New gui_mdi_commands can be broken by another python command without any message.

[c-morley]

Ok Excellent notes. Thank you for testing!
I'll digest the information and probably ask follow up questions.

[zz912]

I don't want to discuss the pros/cons of Bridge now.
Does my Diagram match your PR now?
LCNC_Architecture_C3.drawio