Wind River Support Network

HomeDefectsLIN7-6080
Not to be fixed

LIN7-6080 : WRL LTTng: Bad things happen when trying to enable an UST channel in an active session that's already tracing Kernel

Created: Dec 2, 2014    Updated: Apr 19, 2018
Resolved Date: Apr 17, 2018
Previous ID: WB4-4853
Found In Version: 7.0.0.9
Severity: Standard
Applicable for: Wind River Linux 7
Component/s: Analysis

Description

When trying to enable an UST Channel in an LTTng Tracing Session that's already running, an error message is printed - so far so good. But the problem is that from that point on, any "Refresh" of the LTTng target node leads to errors thus the running tracing session can not be canceled any more from the UI ! - A UI User is completely stuck at this point.

The first error shown is this one:
org.eclipse.core.commands.ExecutionException: Command failed! Command: lttng enable-event lttng_ust_tracef:event,ust_baddr_statedump:soinfo -u  -s context-switches --tracepoint 
Error Ouptut:
Error: Event lttng_ust_tracef:event: Tracing already started (channel channel0, session context-switches)
Error: Event ust_baddr_statedump:soinfo: Tracing already started (channel channel0, session context-switches)
Warning: Some command(s) went wrong
Return Value: 4


	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.executeCommand(LTTngControlService.java:1573)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.executeCommand(LTTngControlService.java:1542)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.enableEvents(LTTngControlService.java:838)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.model.impl.TraceSessionComponent.enableEvents(TraceSessionComponent.java:365)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.handlers.AssignEventHandler$1.run(AssignEventHandler.java:101)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)


Later, following errors are shown:

org.eclipse.core.commands.ExecutionException: Command failed! Command: lttng list context-switches
Error Ouptut:
Error: UST channel not found
Return Value: 59
Tracing session context-switches: [inactive]
    Trace path: /root/lttng-traces/context-switches-20141128-092059
    Live timer interval (usec): 0

=== Domain: Kernel ===

Channels:
-------------
- channel-context-switches: [enabled]

    Attributes:
      overwrite mode: 0
      subbufers size: 262144
      number of subbufers: 4
      switch timer interval: 0
      read timer interval: 200000
      trace file count: 0
      trace file size (bytes): 0
      output: splice()

    Events:
      lttng_statedump_interrupt (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      lttng_statedump_process_state (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      irq_handler_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      irq_handler_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      sched_process_fork (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      sched_switch (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]

=== Domain: UST global ===

Buffer type: per UID

	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.executeCommand(LTTngControlService.java:1573)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.executeCommand(LTTngControlService.java:1542)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.getSession(LTTngControlService.java:197)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.model.impl.TraceSessionComponent.getConfigurationFromNode(TraceSessionComponent.java:284)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.handlers.BaseControlViewHandler$1.run(BaseControlViewHandler.java:83)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)


And finally this one:


org.eclipse.core.commands.ExecutionException: Command failed! Command: lttng list context-switches
Error Ouptut:
Error: UST channel not found
Return Value: 59
Tracing session context-switches: [inactive]
    Trace path: /root/lttng-traces/context-switches-20141128-092059
    Live timer interval (usec): 0

=== Domain: Kernel ===

Channels:
-------------
- channel-context-switches: [enabled]

    Attributes:
      overwrite mode: 0
      subbufers size: 262144
      number of subbufers: 4
      switch timer interval: 0
      read timer interval: 200000
      trace file count: 0
      trace file size (bytes): 0
      output: splice()

    Events:
      lttng_statedump_interrupt (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      lttng_statedump_process_state (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      irq_handler_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      irq_handler_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      sched_process_fork (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]
      sched_switch (loglevel: TRACE_EMERG (0)) (type: tracepoint) [enabled]

=== Domain: UST global ===

Buffer type: per UID

	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.executeCommand(LTTngControlService.java:1573)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.executeCommand(LTTngControlService.java:1542)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.service.LTTngControlService.getSession(LTTngControlService.java:197)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.model.impl.TraceSessionComponent.getConfigurationFromNode(TraceSessionComponent.java:284)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.model.impl.TraceSessionGroup.getSessionsFromNode(TraceSessionGroup.java:119)
	at org.eclipse.linuxtools.internal.lttng2.control.ui.views.model.impl.TargetNodeComponent$3.run(TargetNodeComponent.java:388)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)

Workaround

Disconnect the LTTng Control view, then log in to the target and on the command-line perform

lttng stop context-switches

(where context-switches is the session name, and can be replaced with some other session name). Note that killing processes like lttng-consumerd is not recommended since it looks like they leave state around which makes restarting them impossible without a target reboot.

Steps to Reproduce

1. Connect an LTTng Tracing session, add the "wrsv-context-switches.sh" session events and start tracing
2. Launch /usr/sbin/lemon_server on the target
3. Refresh the LTTng Tracing Node in the Control View
4. Select the new UST node, expand it and right-click > Enable Channel
5. Select the running tracing session and press OK
--> An error dialog shows up
--> The Session nodes in the control view collapse and the "pause" button vanishes
--> It's no longer possible to configure / stop / terminate the tracing session !
Live chat
Online