sysfs.txt 4.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129
  1. SYSFS FILES
  2. For each InfiniBand device, the InfiniBand drivers create the
  3. following files under /sys/class/infiniband/<device name>:
  4. node_type - Node type (CA, switch or router)
  5. node_guid - Node GUID
  6. sys_image_guid - System image GUID
  7. In addition, there is a "ports" subdirectory, with one subdirectory
  8. for each port. For example, if mthca0 is a 2-port HCA, there will
  9. be two directories:
  10. /sys/class/infiniband/mthca0/ports/1
  11. /sys/class/infiniband/mthca0/ports/2
  12. (A switch will only have a single "0" subdirectory for switch port
  13. 0; no subdirectory is created for normal switch ports)
  14. In each port subdirectory, the following files are created:
  15. cap_mask - Port capability mask
  16. lid - Port LID
  17. lid_mask_count - Port LID mask count
  18. rate - Port data rate (active width * active speed)
  19. sm_lid - Subnet manager LID for port's subnet
  20. sm_sl - Subnet manager SL for port's subnet
  21. state - Port state (DOWN, INIT, ARMED, ACTIVE or ACTIVE_DEFER)
  22. phys_state - Port physical state (Sleep, Polling, LinkUp, etc)
  23. There is also a "counters" subdirectory, with files
  24. VL15_dropped
  25. excessive_buffer_overrun_errors
  26. link_downed
  27. link_error_recovery
  28. local_link_integrity_errors
  29. port_rcv_constraint_errors
  30. port_rcv_data
  31. port_rcv_errors
  32. port_rcv_packets
  33. port_rcv_remote_physical_errors
  34. port_rcv_switch_relay_errors
  35. port_xmit_constraint_errors
  36. port_xmit_data
  37. port_xmit_discards
  38. port_xmit_packets
  39. symbol_error
  40. Each of these files contains the corresponding value from the port's
  41. Performance Management PortCounters attribute, as described in
  42. section 16.1.3.5 of the InfiniBand Architecture Specification.
  43. The "pkeys" and "gids" subdirectories contain one file for each
  44. entry in the port's P_Key or GID table respectively. For example,
  45. ports/1/pkeys/10 contains the value at index 10 in port 1's P_Key
  46. table.
  47. There is an optional "hw_counters" subdirectory that may be under either
  48. the parent device or the port subdirectories or both. If present,
  49. there are a list of counters provided by the hardware. They may match
  50. some of the counters in the counters directory, but they often include
  51. many other counters. In addition to the various counters, there will
  52. be a file named "lifespan" that configures how frequently the core
  53. should update the counters when they are being accessed (counters are
  54. not updated if they are not being accessed). The lifespan is in milli-
  55. seconds and defaults to 10 unless set to something else by the driver.
  56. Users may echo a value between 0 - 10000 to the lifespan file to set
  57. the length of time between updates in milliseconds.
  58. MTHCA
  59. The Mellanox HCA driver also creates the files:
  60. hw_rev - Hardware revision number
  61. fw_ver - Firmware version
  62. hca_type - HCA type: "MT23108", "MT25208 (MT23108 compat mode)",
  63. or "MT25208"
  64. HFI1
  65. The hfi1 driver also creates these additional files:
  66. hw_rev - hardware revision
  67. board_id - manufacturing board id
  68. tempsense - thermal sense information
  69. serial - board serial number
  70. nfreectxts - number of free user contexts
  71. nctxts - number of allowed contexts (PSM2)
  72. chip_reset - diagnostic (root only)
  73. boardversion - board version
  74. sdma<N>/ - one directory per sdma engine (0 - 15)
  75. sdma<N>/cpu_list - read-write, list of cpus for user-process to sdma
  76. engine assignment.
  77. sdma<N>/vl - read-only, vl the sdma engine maps to.
  78. The new interface will give the user control on the affinity settings
  79. for the hfi1 device.
  80. As an example, to set an sdma engine irq affinity and thread affinity
  81. of a user processes to use the sdma engine, which is "near" in terms
  82. of NUMA configuration, or physical cpu location, the user will do:
  83. echo "3" > /proc/irq/<N>/smp_affinity_list
  84. echo "4-7" > /sys/devices/.../sdma3/cpu_list
  85. cat /sys/devices/.../sdma3/vl
  86. 0
  87. echo "8" > /proc/irq/<M>/smp_affinity_list
  88. echo "9-12" > /sys/devices/.../sdma4/cpu_list
  89. cat /sys/devices/.../sdma4/vl
  90. 1
  91. to make sure that when a process runs on cpus 4,5,6, or 7,
  92. and uses vl=0, then sdma engine 3 is selected by the driver,
  93. and also the interrupt of the sdma engine 3 is steered to cpu 3.
  94. Similarly, when a process runs on cpus 9,10,11, or 12 and sets vl=1,
  95. then engine 4 will be selected and the irq of the sdma engine 4 is
  96. steered to cpu 8.
  97. This assumes that in the above N is the irq number of "sdma3",
  98. and M is irq number of "sdma4" in the /proc/interrupts file.
  99. ports/1/
  100. CCMgtA/
  101. cc_settings_bin - CCA tables used by PSM2
  102. cc_table_bin
  103. cc_prescan - enable prescaning for faster BECN response
  104. sc2v/ - 32 files (0 - 31) used to translate sl->vl
  105. sl2sc/ - 32 files (0 - 31) used to translate sl->sc
  106. vl2mtu/ - 16 (0 - 15) files used to determine MTU for vl