| Message ID | 20201202113942.27024-1-johan@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | tty: add flag to suppress ready signalling on open | expand |
On 12/2/20, Johan Hovold <johan@kernel.org> wrote: > Also let me know if you prefer to hold this off for 5.12. The change is > minimal, self-contained and low-risk, but it is a new interface and late > in the release cycle as Andy pointed out. Hold off for 5.12? Did you perhaps mean 5.11? I understand how this change may be too late for 5.10, but surely it can go into 5.11 merge window, why hold off for 5.12? M~
On Wed, Dec 02, 2020 at 10:07:58AM -0800, Mychaela Falconia wrote: > On 12/2/20, Johan Hovold <johan@kernel.org> wrote: > > Also let me know if you prefer to hold this off for 5.12. The change is > > minimal, self-contained and low-risk, but it is a new interface and late > > in the release cycle as Andy pointed out. > > Hold off for 5.12? Did you perhaps mean 5.11? I understand how this > change may be too late for 5.10, but surely it can go into 5.11 merge > window, why hold off for 5.12? No, I meant 5.12, even if I still think this could go into 5.11. Generally new features should brew a bit in linux-next. It's mostly bots testing linux-next, and they wouldn't be able to catch anything beyond potential build issues as this feature is off by default anyway. So I don't think that needs to be an issue in this case. But we don't have deadlines and if, say, for one reason or another Greg doesn't have time to review this in the next couple of weeks, we'll just hold it off. Johan
On Wed, Dec 02, 2020 at 12:39:35PM +0100, Johan Hovold wrote: > This series adds a new NORDY port flag to suppress raising the > modem-control lines on open to signal DTE readiness. > > This can be used to implement a NORDY termios control flag to complement > HUPCL, which controls lowering of the modem-control lines on final > close. > > Initially drivers can export the flag through sysfs, which also allows > control over the lines on first open. Such an interface is implemented > for serial core and USB serial. > > The motivation for this is to allow for applications where the DTR and > RTS lines are used for non-standard purposes (e.g. generating power-on > and reset pulses) to open the port without undesirable side effects. > > The final patches enables this flag by default for such a USB serial > device. > > Other examples include HAM-radio devices where DTR and RTS is used for > push-to-talk and continuous-wave control and various Arduino boards > which are reset on open unless a jumper is physically removed. > > Greg, are you ok we me taking this through my tree? I'm planning on some > follow ups to the ftdi driver and the tty/serial changes are fairly > self-contained. > > Also let me know if you prefer to hold this off for 5.12. The change is > minimal, self-contained and low-risk, but it is a new interface and late > in the release cycle as Andy pointed out. I took the first 2 patches now, that was easy :) I think we need more review for the rest of the series. This does change the way serial ports work in a non-traditional way (i.e. using sysfs instead of terminal settings). So I want to get a bunch of people to agree that this is ok to do things this way now before taking this new user-visible api. thanks, greg k-h
On Wed, Dec 09, 2020 at 03:30:33PM +0100, Greg Kroah-Hartman wrote: > On Wed, Dec 02, 2020 at 12:39:35PM +0100, Johan Hovold wrote: > > This series adds a new NORDY port flag to suppress raising the > > modem-control lines on open to signal DTE readiness. > > > > This can be used to implement a NORDY termios control flag to complement > > HUPCL, which controls lowering of the modem-control lines on final > > close. > > > > Initially drivers can export the flag through sysfs, which also allows > > control over the lines on first open. Such an interface is implemented > > for serial core and USB serial. > > Also let me know if you prefer to hold this off for 5.12. The change is > > minimal, self-contained and low-risk, but it is a new interface and late > > in the release cycle as Andy pointed out. > > I took the first 2 patches now, that was easy :) > > I think we need more review for the rest of the series. This does > change the way serial ports work in a non-traditional way (i.e. using > sysfs instead of terminal settings). So I want to get a bunch of people > to agree that this is ok to do things this way now before taking this > new user-visible api. Sounds good, thanks. Johan
Greg K-H wrote: > I think we need more review for the rest of the series. This does > change the way serial ports work in a non-traditional way (i.e. using > sysfs instead of terminal settings). But the problem is that the current status quo is fundamentally broken for those hardware devices in which DTR and/or RTS have been repurposed for something other than modem and flow control. Right now whenever a "cold" (never previously opened) serial port is opened for the first time, that open action immediately and unstoppably asserts both DTR and RTS hardware outputs, without giving userspace any opportunity to say "no, please don't do it". Yes, this behaviour is codified in a bunch of standards that ultimately trace back to 1970s Original UNIX, but just because it is a standard does not make it right - this Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a feature. All operating systems are fundamentally middlemen between applications and hardware. If custom application A is specifically written to work with custom hardware H, it is the operating system's job to allow A to operate on H without unreasonable interference, NOT to tell A and H that they should change their hw and/or app design to fit a given OS worldview. Furthermore, hardware engineers who design custom gadgets have a natural right to reduce hardware cost by employing designs that are simple and elegant in hw terms - they should NOT be forced to make hw designs more complicated, more expensive, or less convenient for end users in order to please an OS. To put it another way, OSes and OS maintainers need to see themselves as working for end users and their hardware and applications, not the other way around. Here is my situation: I needed a little hardware adapter that goes from USB to one UART with full modem and flow control signals (8-wire UART), one more UART with only data leads and no modem or flow control (2-wire UART), and two non-serial control signals that would allow a connected host computer to simulate PWON and RESET pushbutton presses. The simplest and most elegant hardware solution was to use an FT2232x chip from FTDI (either FT2232D or FT2232H, I chose the D version because I didn't need USB high speed), wire up one channel for the 8-wire UART, wire up the other channel for the 2-wire UART, and then repurpose the otherwise unused Channel B RTS & DTR outputs to drive PWON and RESET pushbutton signals via an open drain buffer IC. The entire hardware solution for driving my PWON and RESET signals consists of a single buffer IC that measures 1.00x1.45 mm - an epitome of hardware simplicity and elegance, I would argue. If my target OS was Windows, I wouldn't be having all these problems: under Windows when you open a "cold" serial port (USB-serial converter just plugged in, not previously opened) for the first time, that first-open action does NOT jerk DTR or RTS modem control signals, instead they are left alone - and the application program can then control and configure how they should function subsequently. But Linux is being a bad boy: when the serial port is opened, DTR and RTS are immediately asserted, and there is no way to prevent it other than by applying a local patch to the kernel on your system. I realize that changing the default first-open behaviour to leave DTR and RTS alone instead of asserting them would not be acceptable. There are lots of devices that require DTR and/or RTS to be asserted for serial communication to work, and because the standard (however poorly thought through) behaviour has always been to auto-assert these signals on open, most standard serial applications rely on this automatic assertion - almost no one does an explicit TIOCMBIS of DTR and RTS after open. Thinking as a hardware engineer, to me the most natural solution to this Linux problem is to use a custom USB ID upon which a driver quirk is then conditionalized. All major USB-serial chips feature either a built-in EEPROM or OTP configuration memory, or a provision for connecting a board-level EEPROM. One of the parameters set in this hardware config EEPROM is the USB VID:PID which the USB-serial chip will present itself as. Furthermore, most USB-serial chip vendors (at least FTDI and Silabs for sure) allow their customers (designers of custom hw containing their chips) to obtain a custom PID out of that chip vendor's USB VID, making it very very easy for custom USB-serial boards to have custom USB IDs. So I program the EEPROM on my custom DUART28 board with the custom USB ID which FTDI officially allocated to me (thus this ID code absolutely uniquely identifies my specific hardware and no other), and I prepare a patch for the ftdi_sio driver which I thought would be non-controversial: adding support for my new FT2232D-based device by adding its custom USB ID to the ID code table, along with a quirk that suppresses automatic assertion of DTR & RTS on open for the Channel B ttyUSB device. I thought my patch would be non-controversial: it is simply adding support for a new hardware device which was not previously supported by Linux at all, same as if I were adding an entirely new driver, and has ZERO impact on any other hardware, any other applications, any other users, any other anything - just support for a new hw device with a special quirk and nothing more. But Johan objects to simply merging my ftdi_sio driver patch by itself because it is not a general solution, because it covers only one specific hw device. Johan proposed a more generalized solution that would still satisfy me and the end users of my hardware because in the end it still includes recognition of my custom USB ID, setting the needed quirk flag upon seeing this ID - but now Johan's generalized patch series is not acceptable either? So what gives? At the end of the day we have a situation where the creators of Original UNIX have made a design mistake back in the 1970s, and this design mistake persists to this day in all current Unix-style OSes including Linux. The design mistake in question is unconditionally asserting DTR and RTS upon the first open of a previously cold serial port. This behaviour is harmless for most devices in which DTR and RTS retain their standard RS-232 functions, but it is a killer for those hw devices in which either or both signals have been repurposed for something very different, some other function that does not tolerate unwanted signal assertion. Unix/POSIX/Linux OSes are at fault here, I argue, NOT hardware engineers who build gadgets in which these signals are repurposed. The physical hardware behaviour of a classic old-fashioned RS-232 DTE port is that when the hardware powers up, the output voltage on DTR and RTS lines goes from zero to -12 V, indicating negated state - it never goes toward +12 V corresponding to asserted state, until and unless software takes action to raise those signals. Same deal with modern USB-serial chips: when USB VBUS power is applied, DTR and RTS outputs immediately go CMOS high (still corresponding to negated state), or they remain tristated until enumeration, in which case pull-up resistors solve the problem - but they *never* go into the state of actively driving CMOS low (overriding pull-up resistors) until and unless software sends the necessary USB control pipe command. Hardware engineers have every right to design gadgets that take actions upon asserted DTR/RTS (CMOS low or RS-232 +12V), actions that should not occur when not explicitly requested, and we the hardware engineers feel justified in this stance because the state of CMOS low or RS-232 +12V on DTR and RTS control lines never occurs in the hardware on its own, this state only occurs upon software command - thus we the hardware engineers have the right to ask OS maintainers to provide a way to NOT send that software command until and unless the user's application explicitly requests it. If this design flaw had been noticed back in the 1970s and addressed in some way, we wouldn't be having this discussion. But here we are almost a half-century later, and the problem is still unresolved. So how do we solve this problem now, after having ignored it for 45 y? Simply changing the default action upon first open of a previously cold serial port to not automatically assert DTR & RTS won't work: plenty of devices (classic modems etc) require DTR and/or RTS to be asserted for communication to work, and plenty of applications rely on automatic assertion of these signals on the first open of a previously cold serial port. Adding configurable termios flags etc won't help either: one needs to open a port in order to do ioctls on it, and if that first open still raises DTR & RTS, then users of custom hw are still screwed. Thus we need some out-of-band mechanism. There are only 3 possibilities: 1) Reusing/abusing some open flag like O_DIRECT, perhaps only for the very first open, and then setting a termios or setserial flag to change behaviour on subsequent opens; 2) A sysfs attribute like in Johan's proposed patch series; 3) ID-code-specific quirks in USB-serial chip drivers. At this point an important question needs to be asked. Johan mentioned that the issue I am dealing with (needing some way to suppress unwanted auto-assertion of DTR/RTS) has come up in the past in the context of ham radio devices. Does Johan or anyone else know what their physical form factor and serial interface was/is? Was it true old-fashioned RS-232 such that the device could be connected to "any" serial port, or was it a custom USB-interfaced board with an integrated USB-serial chip? If every non-standard serial port device currently in existence that does not tolerate automatic assertion of DTR/RTS and requires the changes we discussing takes the physical form of a custom circuit board with a USB connector and a built-in USB-serial chip, with all UART signals staying in the CMOS (or more likely LVCMOS) logic voltage level domain without ever becoming old-fashioned RS-232, then the whole argument about supporting "any" serial port becomes moot. In this case the simplest and least invasive solution is to assign custom USB IDs to all such non-standard devices by way of their respective EEPROMs, and support these custom USB ID devices with driver quirks - no need for sysfs, no need for O_DIRECT, no need to touch any kernel code at all outside of individual USB-serial chip drivers. But if there exist some custom hw devices out there that are in the same predicament as my DUART28 adapter, but are different in that they are classic old-fashioned RS-232 rather than integrated USB-serial, with no place to assign a custom USB ID, *then* we need a non-USB-ID- dependent solution such as Johan's sysfs attribute or O_DIRECT. > So I want to get a bunch of people > to agree that this is ok to do things this way now before taking this > new user-visible api. So where and how are we going to find that bunch of people? What happens if no one gives a damn one way or another? Neither I nor Johan are proposing any change whatsoever to any existing behaviour or any existing interfaces - simply providing a new option, such that if you don't care about it and don't set it, absolutely nothing changes and nothing is impacted in any way whatsoever. If the concern is with the new sysfs interface or the proposed O_DIRECT alternative, how about deferring those while allowing specific USB ID support to go in first? Right now there already exists at least one piece of hardware actively supported by its manufacturer (my gadget) that has a custom USB ID and requires the quirk - what is wrong with adding support for this existing specific hw? How about merging Johan's patch that defines the NORDY flag in tty_port, merging the ftdi_sio driver patch setting this flag for my custom USB ID, allowing other hardware engineers in the same boat to submit similar quirk patches for their affected custom hw with custom USB IDs, while deferring the sysfs patches until there is a more pressing need for quirky devices that have no custom USB IDs? Sincerely, Mychaela
On Wed, Dec 09, 2020 at 02:49:45PM -0800, Mychaela Falconia wrote: > Greg K-H wrote: > > > I think we need more review for the rest of the series. This does > > change the way serial ports work in a non-traditional way (i.e. using > > sysfs instead of terminal settings). > > But the problem is that the current status quo is fundamentally broken > for those hardware devices in which DTR and/or RTS have been repurposed > for something other than modem and flow control. Right now whenever a > "cold" (never previously opened) serial port is opened for the first > time, that open action immediately and unstoppably asserts both DTR > and RTS hardware outputs, without giving userspace any opportunity to > say "no, please don't do it". Yes, this behaviour is codified in a > bunch of standards that ultimately trace back to 1970s Original UNIX, > but just because it is a standard does not make it right - this > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a > feature. Thanks for the long response, but I think you have to realize that creating a new api for something that has been "how things work" since the 1970's should not be taken lightly. No matter if it was a bug or not, changing user-visable behavior is not a trivial thing. What we come up with here has to stand the test of time of being able to be supported properly for the next 40+ years. thanks, greg k-h
Hello Mychaela, On 2020-12-09 23:49, Mychaela Falconia wrote: > Greg K-H wrote: > >> I think we need more review for the rest of the series. This does >> change the way serial ports work in a non-traditional way (i.e. using >> sysfs instead of terminal settings). > > But the problem is that the current status quo is fundamentally broken > for those hardware devices in which DTR and/or RTS have been repurposed > for something other than modem and flow control. Right now whenever a > "cold" (never previously opened) serial port is opened for the first > time, that open action immediately and unstoppably asserts both DTR > and RTS hardware outputs, without giving userspace any opportunity to > say "no, please don't do it". Yes, this behaviour is codified in a > bunch of standards that ultimately trace back to 1970s Original UNIX, > but just because it is a standard does not make it right - this > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a > feature. I agree. And an application not configuring the required handshakes, but still relying on them is an equal bug. > But if there exist some custom hw devices out there that are in the > same predicament as my DUART28 adapter, but are different in that they > are classic old-fashioned RS-232 rather than integrated USB-serial, > with no place to assign a custom USB ID, *then* we need a non-USB-ID- > dependent solution such as Johan's sysfs attribute or O_DIRECT. Any device with a classic old-fashioned RS-232 has probably already solved this in another way or is accepted as not working on Linux. And then there is also the device tree (overlay?) through which a quirk like this can be communicated to the kernel driver. Not sure if this could help for a plug-and-play device like on USB. >> So I want to get a bunch of people >> to agree that this is ok to do things this way now before taking this >> new user-visible api. Personally, I would prefer the VID:PID to enforce the quirk and an O_DIRECT (or other) flag used on open() as general backup plan. To me a sysfs solution seems illogical. > If the concern is with the new sysfs interface or the proposed O_DIRECT > alternative, how about deferring those while allowing specific USB ID > support to go in first? Right now there already exists at least one > piece of hardware actively supported by its manufacturer (my gadget) > that has a custom USB ID and requires the quirk - what is wrong with > adding support for this existing specific hw? How about merging > Johan's patch that defines the NORDY flag in tty_port, merging the > ftdi_sio driver patch setting this flag for my custom USB ID, allowing > other hardware engineers in the same boat to submit similar quirk > patches for their affected custom hw with custom USB IDs, while > deferring the sysfs patches until there is a more pressing need for > quirky devices that have no custom USB IDs? > > Sincerely, > Mychaela Again, I agree. Maarten
On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote: > Hello Mychaela, > > On 2020-12-09 23:49, Mychaela Falconia wrote: > > Greg K-H wrote: > > > > > I think we need more review for the rest of the series. This does > > > change the way serial ports work in a non-traditional way (i.e. using > > > sysfs instead of terminal settings). > > > > But the problem is that the current status quo is fundamentally broken > > for those hardware devices in which DTR and/or RTS have been repurposed > > for something other than modem and flow control. Right now whenever a > > "cold" (never previously opened) serial port is opened for the first > > time, that open action immediately and unstoppably asserts both DTR > > and RTS hardware outputs, without giving userspace any opportunity to > > say "no, please don't do it". Yes, this behaviour is codified in a > > bunch of standards that ultimately trace back to 1970s Original UNIX, > > but just because it is a standard does not make it right - this > > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a > > feature. > > I agree. And an application not configuring the required handshakes, but > still relying on them is an equal bug. > > > But if there exist some custom hw devices out there that are in the > > same predicament as my DUART28 adapter, but are different in that they > > are classic old-fashioned RS-232 rather than integrated USB-serial, > > with no place to assign a custom USB ID, *then* we need a non-USB-ID- > > dependent solution such as Johan's sysfs attribute or O_DIRECT. > > Any device with a classic old-fashioned RS-232 has probably already > solved this in another way or is accepted as not working on Linux. > > And then there is also the device tree (overlay?) through which a quirk > like this can be communicated to the kernel driver. Not sure if this > could help for a plug-and-play device like on USB. > > > > So I want to get a bunch of people > > > to agree that this is ok to do things this way now before taking this > > > new user-visible api. > > Personally, I would prefer the VID:PID to enforce the quirk and an > O_DIRECT (or other) flag used on open() as general backup plan. To > me a sysfs solution seems illogical. The "problem" of a vid:pid is that for usb-serial devices, that only describes the device that does the conversion itself, NOT the serial device the converter is plugged into that cares about these types of line-wiggling. Just like you would not want to classify all devices that met the PCI serial class signature for this type of thing either, there is nothing special about USB here other than it happens to be a common transport for these signals these days. thanks, greg k-h
On 2020-12-10 11:50, Greg Kroah-Hartman wrote: > On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote: >> Hello Mychaela, >> >> On 2020-12-09 23:49, Mychaela Falconia wrote: >> > Greg K-H wrote: >> > >> > > I think we need more review for the rest of the series. This does >> > > change the way serial ports work in a non-traditional way (i.e. using >> > > sysfs instead of terminal settings). >> > >> > But the problem is that the current status quo is fundamentally broken >> > for those hardware devices in which DTR and/or RTS have been repurposed >> > for something other than modem and flow control. Right now whenever a >> > "cold" (never previously opened) serial port is opened for the first >> > time, that open action immediately and unstoppably asserts both DTR >> > and RTS hardware outputs, without giving userspace any opportunity to >> > say "no, please don't do it". Yes, this behaviour is codified in a >> > bunch of standards that ultimately trace back to 1970s Original UNIX, >> > but just because it is a standard does not make it right - this >> > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a >> > feature. >> >> I agree. And an application not configuring the required handshakes, >> but >> still relying on them is an equal bug. >> >> > But if there exist some custom hw devices out there that are in the >> > same predicament as my DUART28 adapter, but are different in that they >> > are classic old-fashioned RS-232 rather than integrated USB-serial, >> > with no place to assign a custom USB ID, *then* we need a non-USB-ID- >> > dependent solution such as Johan's sysfs attribute or O_DIRECT. >> >> Any device with a classic old-fashioned RS-232 has probably already >> solved this in another way or is accepted as not working on Linux. >> >> And then there is also the device tree (overlay?) through which a >> quirk >> like this can be communicated to the kernel driver. Not sure if this >> could help for a plug-and-play device like on USB. >> >> > > So I want to get a bunch of people >> > > to agree that this is ok to do things this way now before taking this >> > > new user-visible api. >> >> Personally, I would prefer the VID:PID to enforce the quirk and an >> O_DIRECT (or other) flag used on open() as general backup plan. To >> me a sysfs solution seems illogical. > > The "problem" of a vid:pid is that for usb-serial devices, that only > describes the device that does the conversion itself, NOT the serial > device the converter is plugged into that cares about these types of > line-wiggling. > > Just like you would not want to classify all devices that met the PCI > serial class signature for this type of thing either, there is nothing > special about USB here other than it happens to be a common transport > for these signals these days. > > thanks, > > greg k-h This is true for a generic USB-UART board or cable, but not for a dedicated PCB where both the USB-UART chip and the special connection are implemented and which has a dedicated VID:PID different from any generic one. In this case the VID:PID describes the whole board. If the line-wiggling requirement is created behind some sort of connector (real RS-232 DB9/DB25 or CMOS pin header or whatever) then the problem is the same as for an 8250 on any other bus. For this situation I would prefer the O_DIRECT flag on open(). Maarten
On Thu, Dec 10, 2020 at 01:05:15PM +0100, Maarten Brock wrote: > On 2020-12-10 11:50, Greg Kroah-Hartman wrote: > > On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote: > > > Hello Mychaela, > > > > > > On 2020-12-09 23:49, Mychaela Falconia wrote: > > > > Greg K-H wrote: > > > > > > > > > I think we need more review for the rest of the series. This does > > > > > change the way serial ports work in a non-traditional way (i.e. using > > > > > sysfs instead of terminal settings). > > > > > > > > But the problem is that the current status quo is fundamentally broken > > > > for those hardware devices in which DTR and/or RTS have been repurposed > > > > for something other than modem and flow control. Right now whenever a > > > > "cold" (never previously opened) serial port is opened for the first > > > > time, that open action immediately and unstoppably asserts both DTR > > > > and RTS hardware outputs, without giving userspace any opportunity to > > > > say "no, please don't do it". Yes, this behaviour is codified in a > > > > bunch of standards that ultimately trace back to 1970s Original UNIX, > > > > but just because it is a standard does not make it right - this > > > > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a > > > > feature. > > > > > > I agree. And an application not configuring the required handshakes, > > > but > > > still relying on them is an equal bug. > > > > > > > But if there exist some custom hw devices out there that are in the > > > > same predicament as my DUART28 adapter, but are different in that they > > > > are classic old-fashioned RS-232 rather than integrated USB-serial, > > > > with no place to assign a custom USB ID, *then* we need a non-USB-ID- > > > > dependent solution such as Johan's sysfs attribute or O_DIRECT. > > > > > > Any device with a classic old-fashioned RS-232 has probably already > > > solved this in another way or is accepted as not working on Linux. > > > > > > And then there is also the device tree (overlay?) through which a > > > quirk > > > like this can be communicated to the kernel driver. Not sure if this > > > could help for a plug-and-play device like on USB. > > > > > > > > So I want to get a bunch of people > > > > > to agree that this is ok to do things this way now before taking this > > > > > new user-visible api. > > > > > > Personally, I would prefer the VID:PID to enforce the quirk and an > > > O_DIRECT (or other) flag used on open() as general backup plan. To > > > me a sysfs solution seems illogical. > > > > The "problem" of a vid:pid is that for usb-serial devices, that only > > describes the device that does the conversion itself, NOT the serial > > device the converter is plugged into that cares about these types of > > line-wiggling. > > > > Just like you would not want to classify all devices that met the PCI > > serial class signature for this type of thing either, there is nothing > > special about USB here other than it happens to be a common transport > > for these signals these days. > > > > thanks, > > > > greg k-h > > This is true for a generic USB-UART board or cable, but not for a > dedicated PCB where both the USB-UART chip and the special connection > are implemented and which has a dedicated VID:PID different from any > generic one. In this case the VID:PID describes the whole board. Companies/devices lie about vid:pid all the time, wait until your specific vid:pid is repurposed for some other device and then what happens? :) > If the line-wiggling requirement is created behind some sort of > connector (real RS-232 DB9/DB25 or CMOS pin header or whatever) > then the problem is the same as for an 8250 on any other bus. For > this situation I would prefer the O_DIRECT flag on open(). O_DIRECT is an interesting hack, has anyone seen if it violates the posix rules for us to use it on a character device like this? thanks, greg k-h
Greg K-H wrote: > Thanks for the long response, but I think you have to realize that > creating a new api for something that has been "how things work" since > the 1970's should not be taken lightly. No matter if it was a bug or > not, changing user-visable behavior is not a trivial thing. But I am NOT asking to change any user-visible behaviour on any existing hardware made by anyone other than me, instead I am merely asking to mainline a simple purely additive patch that adds support for my custom FTDI device with a custom USB ID, to spare my users the pain of applying that same patch locally on their own systems. As a secondary objective, I am also contributing my thoughts toward a generic solution for potential other affected devices that don't have the luxury of a custom PCB with a fully embedded USB-serial chip - but that one is more theoretical at the moment. > What we > come up with here has to stand the test of time of being able to be > supported properly for the next 40+ years. And how do we achieve this goal if the total number of people in the world who care in any way at all can be counted on one hand? Maarten Brock wrote: > I agree. And an application not configuring the required handshakes, but > still relying on them is an equal bug. This comment can be interpreted in at least two different ways. Are you referring to: 1) Mainstream existing applications that expect DTR and/or RTS to be asserted on open without doing any explicit TIOCMBIS, or 2) My fc-host-tools programs (fc-loadtool, fc-xram, rvinterf etc) that operate on the second UART of my DUART28C adapter, expect to NOT have auto-assertion of DTR/RTS on open, but rely on my custom USB ID and the ftdi_sio driver patch that goes with it to suppress this auto- assertion, without doing any explicit TIOCMBIC. If you are referring to 1, it is difficult to fault the authors of those applications, as they had every right to depend on the behaviour that had been codified in numerous official standards like POSIX. Or if you are referring to 2, what other choice do I have? With existing unpatched Linux kernels of every currently existing version, it is *impossible* to prevent DTR & RTS auto-assertion immediately on open of a tty device, thus applying a patch to the kernel (or at least to the ftdi_sio driver in my case) is the *only* way to make my hardware work with Linux. Doing a TIOCMBIC after open won't help, as it will be too late if the kernel already asserted DTR & RTS and thus caused an unwanted deep reset. > Any device with a classic old-fashioned RS-232 has probably already > solved this in another way or is accepted as not working on Linux. If someone built such a device for their own personal enjoyment rather than for commercial sale, and needed to get it working with Linux, I suspect that person most likely applied a local patch to the kernel on their own system, likely implementing something similar to what is being discussed in this thread. > And then there is also the device tree (overlay?) through which a quirk > like this can be communicated to the kernel driver. This method should indeed work for embedded boards that feature some kind of SoC where one or more native UARTs on that SoC need the quirk applied to them. But I don't have any such use cases, so let's defer this approach until and unless there is at least one person in the world who does. > Not sure if this > could help for a plug-and-play device like on USB. I don't see how this method can be useful at all for devices that connect to a standard (non-embedded) PC or laptop, whether USB or RS-232. > Personally, I would prefer the VID:PID to enforce the quirk and an > O_DIRECT (or other) flag used on open() as general backup plan. To > me a sysfs solution seems illogical. A sysfs solution could work as a sort of poor man's substitute for a VID:PID-driven quirk: instead of a driver quirk in the kernel, there is a udev rule that detects a particular USB-serial device (perhaps based on textual manuf/product strings as opposed to VID:PID) and sets the needed sysfs flag. But then if we are talking about a special USB device as opposed to generic serial as in RS-232 etc, then I argue for a driver quirk: if the device has a custom VID:PID, a patch to the driver is needed in any case just to recognize that custom ID, so why not support the custom hw device properly by setting the quirk bit right there and then? Seen in this light, the sysfs approach indeed makes little sense. OTOH, if we are talking about RS-232 or similarly interfaced devices which the user plugs into any random serial port (PC native, or a random off-the-shelf USB-serial cable), then there is really nothing that a udev rule can key onto, it is just a user plugging in some serial device and then running custom userspace apps on it. In this case asking the user to 'echo' something from the shell into /sys/blah prior to running her userspace app seems illogical indeed, and asking userspace app programmers to implement an equivalent sysfs write internally is equally awkward. For this non-custom-USB-ID scenario I thus agree that the O_DIRECT approach would be better - in this case the userspace app programmer simply needs to add this one flag to their open call, a trivial one line change. > > [Greg's objection to USB VID:PID quirks] > > This is true for a generic USB-UART board or cable, but not for a > dedicated PCB where both the USB-UART chip and the special connection > are implemented and which has a dedicated VID:PID different from any > generic one. In this case the VID:PID describes the whole board. Maarten's understanding is fully correct here: my DUART28C adapter is a custom PCB that contains both the FT2232D USB-serial chip *and* the special wiring whereby Channel B RTS & DTR outputs are repurposed into CTL1 & CTL2 non-serial-related open drain control outputs, and my custom VID:PID of 0x0403:0x7152 uniquely identifies this complete custom-functionality PCB, not just the FT2232D part of it. > If the line-wiggling requirement is created behind some sort of > connector (real RS-232 DB9/DB25 or CMOS pin header or whatever) > then the problem is the same as for an 8250 on any other bus. For > this situation I would prefer the O_DIRECT flag on open(). I agree. Greg K-H wrote: > Companies/devices lie about vid:pid all the time, Wait, are you accusing *me* of lying? PID range 0x7150 through 0x7157 out of FTDI's VID 0x0403 has been officially allocated by FTDI (please feel free to confirm with FTDI, no need to take my word blindly) to Falconia Partners LLC, which is *my* company - Falconia is my last name. Therefore, any accusation of lying in connection with any VID:PID in this range, including DUART28C VID:PID of 0x0403:0x7152, is an accusation of lying against me personally, which I take very seriously. > wait until your > specific vid:pid is repurposed for some other device and then what > happens? :) Let's break it down: 1) I do have an idea for a future hw product that could indeed reuse DUART28C VID:PID - however, the reason for reusing the same USB ID is that the potential product in question would have exactly the same wiring inside and require the exact same handling from Linux, i.e., suppress automatic DTR/RTS assertion on Channel B, but retain standard behaviour on Channel A. 2) If I were to reuse the same USB ID for a different hw product that should NOT receive the exact same quirk treatment, it would be 100% my fault and I would have no right to run to OS maintainers complaining, or even to sell such product, I would argue. 3) If some other party illegally squats on a PID out of FTDI's VID range which FTDI officially allocated to me, and then comes to Linux maintainers with a complaint however many years from now, whoever will be the maintainer at that time will be able to check with FTDI, get an official answer as to whom that PID rightfully belongs to, and that will be the resolution. > O_DIRECT is an interesting hack, has anyone seen if it violates the > posix rules for us to use it on a character device like this? According to open(2) Linux man page, O_DIRECT does not come from POSIX at all, instead it is specific to Linux, FreeBSD and SGI IRIX. Thus it seems like there aren't any POSIX rules to be violated here. If we go with O_DIRECT, what semantics are we going to implement? There are 3 possibilities that come to mind most readily: 1) O_DIRECT applies only to the open call in which this flag is set, and suppresses DTR/RTS assertion on that open. If someone needs to do multiple opens with DTR/RTS suppression being required every time, then they need to include O_DIRECT every time. 2) O_DIRECT applies not only immediately, but also sets a latched flag whereby all subsequent opens continue to suppress auto-assertion without requiring O_DIRECT every time. This approach by itself runs counter to the generic Unix way of doing things, but it may be OK if there is also some ioctl to explicitly set or clear the latched flag. 3) O_DIRECT applies only to the open call in which it is set, no built-in latching, but there is also some ioctl to control a flag enabling or disabling DTR/RTS auto-assertion on subsequent opens. My vote would be to implement 1 first, for reasons of utmost simplicity, both interface and implementation, and then possibly implement 3 at some later time if and when someone actually needs that extra bit of complexity. A strong argument can be made that if someone repurposed DTR and/or RTS signals on a serial port for some very different non-standard purpose, then they will almost certainly be writing their own custom userspace sw to talk to that serial port, as opposed to using standard existing userspace sw, hence asking them to include a non-standard flag on every open of their serial port shouldn't be too burdensome. M~
On 10. 12. 20, 19:59, Mychaela Falconia wrote: >> O_DIRECT is an interesting hack, has anyone seen if it violates the >> posix rules for us to use it on a character device like this? > > According to open(2) Linux man page, O_DIRECT does not come from POSIX > at all, instead it is specific to Linux, FreeBSD and SGI IRIX. Thus > it seems like there aren't any POSIX rules to be violated here. > > If we go with O_DIRECT, what semantics are we going to implement? > There are 3 possibilities that come to mind most readily: > > 1) O_DIRECT applies only to the open call in which this flag is set, > and suppresses DTR/RTS assertion on that open. If someone needs to do > multiple opens with DTR/RTS suppression being required every time, > then they need to include O_DIRECT every time. > > 2) O_DIRECT applies not only immediately, but also sets a latched flag > whereby all subsequent opens continue to suppress auto-assertion > without requiring O_DIRECT every time. This approach by itself runs > counter to the generic Unix way of doing things, but it may be OK if > there is also some ioctl to explicitly set or clear the latched flag. > > 3) O_DIRECT applies only to the open call in which it is set, no > built-in latching, but there is also some ioctl to control a flag > enabling or disabling DTR/RTS auto-assertion on subsequent opens. 3) -- to allow standard tools to work on the device after the quirk is set up once. thanks,
On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote: > Hello Mychaela, > > On 2020-12-09 23:49, Mychaela Falconia wrote: > > But the problem is that the current status quo is fundamentally broken > > for those hardware devices in which DTR and/or RTS have been repurposed > > for something other than modem and flow control. Right now whenever a > > "cold" (never previously opened) serial port is opened for the first > > time, that open action immediately and unstoppably asserts both DTR > > and RTS hardware outputs, without giving userspace any opportunity to > > say "no, please don't do it". Yes, this behaviour is codified in a > > bunch of standards that ultimately trace back to 1970s Original UNIX, > > but just because it is a standard does not make it right - this > > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a > > feature. > > I agree. And an application not configuring the required handshakes, but > still relying on them is an equal bug. Ehh, don't be silly. Relying on having Linux assert DTR/RTS on open is in no way a bug; it's the standard behaviour that we must continue to support unless explicitly overridden. Period. > > But if there exist some custom hw devices out there that are in the > > same predicament as my DUART28 adapter, but are different in that they > > are classic old-fashioned RS-232 rather than integrated USB-serial, > > with no place to assign a custom USB ID, *then* we need a non-USB-ID- > > dependent solution such as Johan's sysfs attribute or O_DIRECT. > > Any device with a classic old-fashioned RS-232 has probably already > solved this in another way or is accepted as not working on Linux. > > And then there is also the device tree (overlay?) through which a quirk > like this can be communicated to the kernel driver. Not sure if this > could help for a plug-and-play device like on USB. Not every system use devicetree and no, it doesn't help with hotpluggable buses either. Johan
On Thu, Dec 10, 2020 at 02:01:30PM +0100, Greg Kroah-Hartman wrote: > O_DIRECT is an interesting hack, has anyone seen if it violates the > posix rules for us to use it on a character device like this? Jiri only mentioned O_DIRECT as an example of a flag which we might be able to repurpose/abuse for this. O_DIRECT is linux-specific, not in POSIX, so we'd still end up with a Linux-specific interface if we were to take this route. Johan
On 2020-12-10 19:59, Mychaela Falconia wrote: > Maarten Brock wrote: > >> I agree. And an application not configuring the required handshakes, >> but >> still relying on them is an equal bug. > > This comment can be interpreted in at least two different ways. Are > you referring to: > > 1) Mainstream existing applications that expect DTR and/or RTS to be > asserted on open without doing any explicit TIOCMBIS, > > or > > 2) My fc-host-tools programs (fc-loadtool, fc-xram, rvinterf etc) that > operate on the second UART of my DUART28C adapter, expect to NOT have > auto-assertion of DTR/RTS on open, but rely on my custom USB ID and > the ftdi_sio driver patch that goes with it to suppress this auto- > assertion, without doing any explicit TIOCMBIC. > > If you are referring to 1, it is difficult to fault the authors of > those applications, as they had every right to depend on the behaviour > that had been codified in numerous official standards like POSIX. Or > if you are referring to 2, what other choice do I have? With existing > unpatched Linux kernels of every currently existing version, it is > *impossible* to prevent DTR & RTS auto-assertion immediately on open > of a tty device, thus applying a patch to the kernel (or at least to > the ftdi_sio driver in my case) is the *only* way to make my hardware > work with Linux. Doing a TIOCMBIC after open won't help, as it will > be too late if the kernel already asserted DTR & RTS and thus caused > an unwanted deep reset. I was referring to 1) And I still think that an application that *relies* on handshakes working should configure the handshakes, even if POSIX promises they should be set up. >> Any device with a classic old-fashioned RS-232 has probably already >> solved this in another way or is accepted as not working on Linux. > > If someone built such a device for their own personal enjoyment rather > than for commercial sale, and needed to get it working with Linux, I > suspect that person most likely applied a local patch to the kernel > on their own system, likely implementing something similar to what is > being discussed in this thread. A person might have done that, a company will probably just not support Linux or do a redesign with a different solution. >> Personally, I would prefer the VID:PID to enforce the quirk and an >> O_DIRECT (or other) flag used on open() as general backup plan. To >> me a sysfs solution seems illogical. > > A sysfs solution could work as a sort of poor man's substitute for a > VID:PID-driven quirk: instead of a driver quirk in the kernel, there > is a udev rule that detects a particular USB-serial device (perhaps > based on textual manuf/product strings as opposed to VID:PID) and sets > the needed sysfs flag. But then if we are talking about a special > USB device as opposed to generic serial as in RS-232 etc, then I argue > for a driver quirk: if the device has a custom VID:PID, a patch to the > driver is needed in any case just to recognize that custom ID, so why > not support the custom hw device properly by setting the quirk bit > right there and then? Seen in this light, the sysfs approach indeed > makes little sense. > > OTOH, if we are talking about RS-232 or similarly interfaced devices > which the user plugs into any random serial port (PC native, or a > random off-the-shelf USB-serial cable), then there is really nothing > that a udev rule can key onto, it is just a user plugging in some > serial device and then running custom userspace apps on it. In this > case asking the user to 'echo' something from the shell into /sys/blah > prior to running her userspace app seems illogical indeed, and asking > userspace app programmers to implement an equivalent sysfs write > internally is equally awkward. For this non-custom-USB-ID scenario I > thus agree that the O_DIRECT approach would be better - in this case > the userspace app programmer simply needs to add this one flag to > their open call, a trivial one line change. Or use your option 3) mentioned below: open with O_DIRECT, use ioctl to set the sticky flag and close before starting the application. >> O_DIRECT is an interesting hack, has anyone seen if it violates the >> posix rules for us to use it on a character device like this? > > According to open(2) Linux man page, O_DIRECT does not come from POSIX > at all, instead it is specific to Linux, FreeBSD and SGI IRIX. Thus > it seems like there aren't any POSIX rules to be violated here. > > If we go with O_DIRECT, what semantics are we going to implement? > There are 3 possibilities that come to mind most readily: > > 1) O_DIRECT applies only to the open call in which this flag is set, > and suppresses DTR/RTS assertion on that open. If someone needs to do > multiple opens with DTR/RTS suppression being required every time, > then they need to include O_DIRECT every time. > > 2) O_DIRECT applies not only immediately, but also sets a latched flag > whereby all subsequent opens continue to suppress auto-assertion > without requiring O_DIRECT every time. This approach by itself runs > counter to the generic Unix way of doing things, but it may be OK if > there is also some ioctl to explicitly set or clear the latched flag. > > 3) O_DIRECT applies only to the open call in which it is set, no > built-in latching, but there is also some ioctl to control a flag > enabling or disabling DTR/RTS auto-assertion on subsequent opens. > > My vote would be to implement 1 first, for reasons of utmost > simplicity, both interface and implementation, and then possibly > implement 3 at some later time if and when someone actually needs that > extra bit of complexity. > > A strong argument can be made that if someone repurposed DTR and/or > RTS signals on a serial port for some very different non-standard > purpose, then they will almost certainly be writing their own custom > userspace sw to talk to that serial port, as opposed to using standard > existing userspace sw, hence asking them to include a non-standard > flag on every open of their serial port shouldn't be too burdensome. > > M~ Option 3) looks best to me. Maarten
On Thu, Dec 10, 2020 at 10:59:41AM -0800, Mychaela Falconia wrote: > Greg K-H wrote: > > > Companies/devices lie about vid:pid all the time, > > Wait, are you accusing *me* of lying? PID range 0x7150 through 0x7157 > out of FTDI's VID 0x0403 has been officially allocated by FTDI (please > feel free to confirm with FTDI, no need to take my word blindly) to > Falconia Partners LLC, which is *my* company - Falconia is my last > name. Therefore, any accusation of lying in connection with any > VID:PID in this range, including DUART28C VID:PID of 0x0403:0x7152, is > an accusation of lying against me personally, which I take very > seriously. I am not saying you are using an "invalid id", or copying anything, see below: > > wait until your > > specific vid:pid is repurposed for some other device and then what > > happens? :) > > Let's break it down: > > 1) I do have an idea for a future hw product that could indeed reuse > DUART28C VID:PID - however, the reason for reusing the same USB ID is > that the potential product in question would have exactly the same > wiring inside and require the exact same handling from Linux, i.e., > suppress automatic DTR/RTS assertion on Channel B, but retain standard > behaviour on Channel A. > > 2) If I were to reuse the same USB ID for a different hw product that > should NOT receive the exact same quirk treatment, it would be 100% my > fault and I would have no right to run to OS maintainers complaining, > or even to sell such product, I would argue. > > 3) If some other party illegally squats on a PID out of FTDI's VID > range which FTDI officially allocated to me, and then comes to Linux > maintainers with a complaint however many years from now, whoever will > be the maintainer at that time will be able to check with FTDI, get an > official answer as to whom that PID rightfully belongs to, and that > will be the resolution. We see devices that are "obviously" not the real vid/pid all the time in the wild. There's nothing "illegal" about another company using your vid/pid, look at all of the ones out there already that use the FTDI vendor id yet are "clones", same with pl2303 devices. We also have fuzzing devices that spoof vid/pid pairs in order to test kernel code, as well as being used as malicious devices to hack systems or do other "fun" things. Blindly trusting these numbers are something we can no longer do. The point being, while it is nice to trigger off of these values, beware that it is not the only way that something like a userspace visable change should trigger off of because this is something that a user wants to have happen. This also makes it easier as it "should" work for all serial devices, and not be tied to specific hardware ids, requiring kernel updates for new devices that want to do this. hope this helps explain this a bit more. > > O_DIRECT is an interesting hack, has anyone seen if it violates the > > posix rules for us to use it on a character device like this? > > According to open(2) Linux man page, O_DIRECT does not come from POSIX > at all, instead it is specific to Linux, FreeBSD and SGI IRIX. Thus > it seems like there aren't any POSIX rules to be violated here. Ah, for some reason I thought that newer POSIX releases finally defined this, as vendors wanted that written down so they could "rely" on it. Maybe not... thanks, greg k-h
On Fri, Dec 11, 2020 at 09:46:54AM +0100, Jiri Slaby wrote: > On 10. 12. 20, 19:59, Mychaela Falconia wrote: > > > O_DIRECT is an interesting hack, has anyone seen if it violates the > > > posix rules for us to use it on a character device like this? > > > > According to open(2) Linux man page, O_DIRECT does not come from POSIX > > at all, instead it is specific to Linux, FreeBSD and SGI IRIX. Thus > > it seems like there aren't any POSIX rules to be violated here. > > > > If we go with O_DIRECT, what semantics are we going to implement? > > There are 3 possibilities that come to mind most readily: > > > > 1) O_DIRECT applies only to the open call in which this flag is set, > > and suppresses DTR/RTS assertion on that open. If someone needs to do > > multiple opens with DTR/RTS suppression being required every time, > > then they need to include O_DIRECT every time. > > > > 2) O_DIRECT applies not only immediately, but also sets a latched flag > > whereby all subsequent opens continue to suppress auto-assertion > > without requiring O_DIRECT every time. This approach by itself runs > > counter to the generic Unix way of doing things, but it may be OK if > > there is also some ioctl to explicitly set or clear the latched flag. > > > > 3) O_DIRECT applies only to the open call in which it is set, no > > built-in latching, but there is also some ioctl to control a flag > > enabling or disabling DTR/RTS auto-assertion on subsequent opens. > > 3) -- to allow standard tools to work on the device after the quirk is set > up once. I'm lost, what do you mean here? thanks, greg k-h
On Thu, Dec 10, 2020 at 10:59:41AM -0800, Mychaela Falconia wrote: > Maarten Brock wrote: > > Personally, I would prefer the VID:PID to enforce the quirk and an > > O_DIRECT (or other) flag used on open() as general backup plan. To > > me a sysfs solution seems illogical. > > A sysfs solution could work as a sort of poor man's substitute for a > VID:PID-driven quirk: instead of a driver quirk in the kernel, there > is a udev rule that detects a particular USB-serial device (perhaps > based on textual manuf/product strings as opposed to VID:PID) and sets > the needed sysfs flag. But then if we are talking about a special > USB device as opposed to generic serial as in RS-232 etc, then I argue > for a driver quirk: if the device has a custom VID:PID, a patch to the > driver is needed in any case just to recognize that custom ID, so why > not support the custom hw device properly by setting the quirk bit > right there and then? Seen in this light, the sysfs approach indeed > makes little sense. > > OTOH, if we are talking about RS-232 or similarly interfaced devices > which the user plugs into any random serial port (PC native, or a > random off-the-shelf USB-serial cable), then there is really nothing > that a udev rule can key onto, it is just a user plugging in some > serial device and then running custom userspace apps on it. In this > case asking the user to 'echo' something from the shell into /sys/blah > prior to running her userspace app seems illogical indeed, and asking > userspace app programmers to implement an equivalent sysfs write > internally is equally awkward. For this non-custom-USB-ID scenario I > thus agree that the O_DIRECT approach would be better - in this case > the userspace app programmer simply needs to add this one flag to > their open call, a trivial one line change. My reasons for proposing the NORDY sysfs interface *and* termios flag are as follows: - The change should be minimal, but still support non-standard use of the modem control signals. - The interface should be compatible with using standard tools (e.g. echo and cat). - We control all other aspects of this (e.g. HUPCL) using termios and should continue to do so. - We cannot change the default behaviour so this NORDY flag needs to be disabled by default (i.e. hence the negation). - You cannot generally rely on the state of these lines before opening the port at least once, but for applications where this is possible and where even toggling them once is undesirable we *can* provide some out-of-band mechanism to change the default state of the NORDY flag (but we could also ignore this use case, keeping the status quo). - This could be an interface to control just the initial state of this flag after probe() or it can be used in parallel with the termios interface. The latter is what is implemented here. - As a bonus, using sysfs for this allows this feature to be used also before NORDY support has been added to the c libraries. - A sysfs interface also integrates well with udev and allows custom hardware that depend on this to be setup up once at boot regardless of whether an RS-232 or USB-serial interface is used. Note that one of the BSDs recently added a termios flag with the same semantics as the proposed NORDY which seems to suggest that this is interface is indeed a natural one. Side note: Also the Windows API has a setting for the state of these lines *after* open (i.e. similar to a termios flag), and there are reports of Windows users not expecting the lines to be raised on first open (and behaviour changing between OS releases). For FTDI devices there appears to be some driver-specific out-of-band mechanism in the system properties for setting the default behaviour. Perhaps there's also some general mechanism, I have no idea, I've never used this API. But at least it doesn't seem to be as simple as Mychaela suggested earlier, that Win32 got this right, for example, by never raising the lines on open (by default). > Greg K-H wrote: > > > Companies/devices lie about vid:pid all the time, > > Wait, are you accusing *me* of lying? PID range 0x7150 through 0x7157 > out of FTDI's VID 0x0403 has been officially allocated by FTDI (please > feel free to confirm with FTDI, no need to take my word blindly) to > Falconia Partners LLC, which is *my* company - Falconia is my last > name. Therefore, any accusation of lying in connection with any > VID:PID in this range, including DUART28C VID:PID of 0x0403:0x7152, is > an accusation of lying against me personally, which I take very > seriously. Just chill, and then re-read what Greg wrote. > > wait until your > > specific vid:pid is repurposed for some other device and then what > > happens? :) > > O_DIRECT is an interesting hack, has anyone seen if it violates the > > posix rules for us to use it on a character device like this? > > According to open(2) Linux man page, O_DIRECT does not come from POSIX > at all, instead it is specific to Linux, FreeBSD and SGI IRIX. Thus > it seems like there aren't any POSIX rules to be violated here. No, but we'd still be repurposing a flag which is used for something entirely different. > If we go with O_DIRECT, what semantics are we going to implement? > There are 3 possibilities that come to mind most readily: > > 1) O_DIRECT applies only to the open call in which this flag is set, > and suppresses DTR/RTS assertion on that open. If someone needs to do > multiple opens with DTR/RTS suppression being required every time, > then they need to include O_DIRECT every time. This precludes using standard tools like echo/cat and terminal programs. > 2) O_DIRECT applies not only immediately, but also sets a latched flag > whereby all subsequent opens continue to suppress auto-assertion > without requiring O_DIRECT every time. This approach by itself runs > counter to the generic Unix way of doing things, but it may be OK if > there is also some ioctl to explicitly set or clear the latched flag. Yeah, this is the worst option; you cannot even disable this feature unless you provide an ioctl interface as well. > 3) O_DIRECT applies only to the open call in which it is set, no > built-in latching, but there is also some ioctl to control a flag > enabling or disabling DTR/RTS auto-assertion on subsequent opens. This would be my choice *if* we go this route. > My vote would be to implement 1 first, for reasons of utmost > simplicity, both interface and implementation, and then possibly > implement 3 at some later time if and when someone actually needs that > extra bit of complexity. Option 1 is not generic enough, and essentially only allows for your very specific use case (and not, for example, to suppress reset on open when using various embedded boards with terminal programs/echo/cat). > A strong argument can be made that if someone repurposed DTR and/or > RTS signals on a serial port for some very different non-standard > purpose, then they will almost certainly be writing their own custom > userspace sw to talk to that serial port, as opposed to using standard > existing userspace sw, hence asking them to include a non-standard > flag on every open of their serial port shouldn't be too burdensome. Not true, consider the various Arduino tinkerers, they may not even have designed the reset circuit themselves. Johan
Greg K-H wrote: > We see devices that are "obviously" not the real vid/pid all the time in > the wild. There's nothing "illegal" about another company using your > vid/pid, look at all of the ones out there already that use the FTDI > vendor id yet are "clones", same with pl2303 devices. But are those reusers of someone else's VID or PID coming to Linux kernel maintainers with requests to modify ftdi_sio or pl2303 drivers to work with their clones? Do you ever see LKML posts along the lines of "Hi, I am so and so from such and such company, we are not FTDI but we reuse FTDI's VID and PIDs, but our clone chip does not match the original and we need to modify the ftdi_sio driver to work with our poor man's clone chip" - do reusers of someone else's VID or PID come here with such requests? Let's apply this logic to the very specific example of USB VID:PID 0x0403:0x7152. Right now I am asking you, esteemed maintainers, to accept my patch to ftdi_sio adding support for this new custom ID along with the necessary quirk. If my patch were accepted and then someone else squats on this USB ID, how would it be a problem for anyone other than that squatter? Are you expecting that someone is going to file a bug report saying "I am using 0x0403:0x7152 for my own USB device, but there is a quirk for this ID in ftdi_sio, and I don't want that quirk for MY device using that ID" - is this what you are anticipating? If that scenario were to occur, how could it possibly be right to choose that hypothetical complainant's interest over mine if one can trivially check with FTDI and confirm that 0x0403:0x7152 is officially allocated to Falconia and not to the other contesting party? > We also have fuzzing devices that spoof vid/pid pairs in order to test > kernel code, as well as being used as malicious devices to hack systems > or do other "fun" things. And where is the harm? If my patch adding ftdi_sio support for 0x0403:0x7152 with the special quirk I am asking for were to be accepted, and then someone presents this USB ID to a target system as a form of hack or fuzzing test, where is the harm? If anything, such action would be a good thing, as it would exercise the quirk function and ensure that it doesn't crash the kernel or anything like that. Yes, the quirk flag will be set on that ttyUSB device - but if they are deliberately presenting a USB ID whose owner said "this ID means that this quirk is needed", then how is it a bad thing to apply the quirk when that ID is detected, fuzzing/hacking or not? > Blindly trusting these numbers are something > we can no longer do. Please clarify what you mean by "blindly trusting" in this very specific context. Suppose some company makes an entirely new USB- interfaced chip, they assign it a unique ID (obtained officially through proper channels, not taking squatting on else's), and they submit a brand new Linux driver for their brand new chip, a driver that naturally binds to that chip's new unique ID - are you going to reject that driver and keep the new chip forever unsupported by Linux because you are concerned that someone else will reuse that same USB ID for something completely different? How is my case any different? The only difference is that in my case the custom hw product is a board and not a chip - but how are board-level designs any less worthy of mainline Linux support than chip-level ones? Johan Hovold wrote: > My reasons for proposing the NORDY sysfs interface *and* termios flag > are as follows: The patch series presented so far provides the sysfs interface and the ftdi_sio ID-code-specific quirk for DUART28C (which would make me happy), but does not add any new termios flag. Are you planning to present a new patch series that also adds the new termios flag? > You cannot generally rely on the state of these lines before opening > the port at least once, I disagree. Manufacturers of USB-serial chips like FT232x/FT2232x *guarantee* (see FTDI's AN 184) that their DTR/RTS CMOS outputs *will not* drive low (they will either drive high or be tristated) until and unless an explicit command comes in on the USB control pipe to change those signals to the "on" state of CMOS low output. Therefore, hardware engineers have a right to demand from OS maintainers that OSes (who are middlemen between applications and hw) provide a way to NOT issue that particular USB control pipe command if it is not wanted by the application. > but for applications where this is possible > and where even toggling them once is undesirable s/undesirable/killer/, for the specific case of UART Channel B on FreeCalypso DUART28C, the custom board with the custom USB ID at the crux of my quest. > we *can* provide > some out-of-band mechanism to change the default state of the NORDY > flag (but we could also ignore this use case, keeping the status > quo). In the case of the parenthetical option, are you basically saying "screw you" to me and my users, refusing to mainline our zero-effect- on-others USB ID-driven quirk patch, the one without which the device cannot work with Linux? > This could be an interface to control just the initial state of this > flag after probe() or it can be used in parallel with the termios > interface. The latter is what is implemented here. My concern with the termios flag idea is that in the absence of a specific proposed patch to review, I don't have a clear view of exactly where this new flag will be inserted, and I have a concern about this new flag breaking things. Right now my custom userspace programs initialize termios like this: #include <asm/ioctls.h> #include <asm/termbits.h> ... struct termios2 target_termios; target_termios.c_iflag = IGNBRK; target_termios.c_oflag = 0; target_termios.c_cflag = br->termios_code | CLOCAL|HUPCL|CREAD|CS8; target_termios.c_lflag = 0; target_termios.c_cc[VMIN] = 1; target_termios.c_cc[VTIME] = 0; target_termios.c_ispeed = br->nonstd_speed; target_termios.c_ospeed = br->nonstd_speed; if (ioctl(target_fd, TCSETSF2, &target_termios) < 0) { perror("TCSETSF2"); exit(1); } br->termios_code will be codes like B115200 for standard baud rates, or BOTHER for GSM-specific baud rates like 812500, with the actual bps number going into br->nonstd_speed. You mention C libraries below, and there is an important lesson to be learned here: as I understand it, support for non-standard baud rates via BOTHER was added to Linux as in kernel ages ago (at the urging of embedded systems folks like me), but libc people (whichever libc it is, I don't care for their politics) apparently never got the memo, and it appears to be impossible to set non-standard serial baud rates using standard libc headers and termios APIs. It appears that the *only* way to set a non-standard serial port baud rate from a lowly Linux userspace app is to use <asm/xxx.h> kernel headers and raw ioctls - and as I quickly found out, the necessary <asm/xxx.h> headers *conflict* with standard libc ones, hence the necessary code needs to be moved out into its own compilation unit that doesn't need much in the way of other libc headers. Back to your proposed new termios flag: are you thinking about adding it to c_cflag, where CLOCAL and HUPCL currently live? If so, it would be pretty much impossible for userspace code like mine to not clear this new flag. You could argue that my approach of absolutely setting all termios fields is wrong, that I should read the previous termios state and make my desired changes, but considering that most termios flags are just unwanted noise for raw byte I/O applications, approaches like mine (init termios from scratch) are probably commonplace. OK, I could read previous termios just to read and preserve the NORDY flag - but how would I do it when it will be years (if ever) before the new flag definition appears in userspace-available headers? So here is my concern: suppose that the "generic" solution you guys end up implementing and merging will revolve around a new termios flag added to c_cflag or some similar place, along with a sysfs interface, and maybe even an ftdi_sio driver quirk to set this flag automatically when DUART28C custom USB ID is detected. The first open will be good - but the termios flag will get reset, and breakage will occur with subsequent runs of the same userspace programs messing up the hw. I suppose I could implement a workaround by adding code to my FC host tools userspace sw suite to always do a sysfs write before opening the serial port, but it certainly isn't anywhere near clean. If you still insist on a termios flag rather than just sysfs, one possible clean solution would be to have two separate flags in the kernel's internal data structures: the new termios flag would end up inside struct ktermios in the kernel, whereas the flag manipulated via sysfs or set in my DUART28C USB ID quirk goes into struct tty_port iflags like in Johan's current patch. If either flag is set, suppress automatic DTR & RTS assertion on open. Alternatively, put the new flag under getserial/setserial Linux-specific ioctls, rather than termios - then it won't get inadvertently cleared so easily. > As a bonus, using sysfs for this allows this feature to be used also > before NORDY support has been added to the c libraries. This part is absolutely crucial: the new kernel feature will be essentially useless if the lack of support in userspace-available headers makes it inaccessible. For devices other than my specific DUART28C which I still argue should be covered by an ftdi_sio driver quirk, sysfs is the only viable option for immediate relief of users' suffering. Oh, and here is another reason why a USB-ID-driven ftdi_sio quirk patch is pretty much an absolute requirement for me: even if some generic solution that works for "any" serial port were to be implemented in current bleeding edge mainline, it will be years before these new kernels percolate to end users to a sufficient degree to be considered ubiquitous. Thus for the foreseeable future, I *must* support users of old kernels, and because the necessary feature does not exist there, directing users to apply a local patch is the only option. And if I am telling users to apply a local patch to their kernels, that patch needs to be minimally-invasive-surgical, self-contained entirely within the ftdi_sio driver and not touching anything outside of it, and as easy as possible to apply to a very wide range of old kernel versions. Whatever solution you guys are going to come up with in terms of new termios flags and/or sysfs, it will be infeasible to package it in a form of a retro-patch which any Joe End User can trivially apply to his or her old system running whatever random old kernel version - but my self-contained ftdi_sio patch adding support for my custom USB ID with the needed quirk *is* trivially applicable by end users on just about any old system. Thus I am begging and pleading with you to accept my DUART28C USB ID quirk *in addition* to whatever "generic" or "works for anyone" solution you esteemed gentlemen come up with. > Note that one of the BSDs recently added a termios flag with the same > semantics as the proposed NORDY which seems to suggest that this is > interface is indeed a natural one. Would you happen to have a link or at least some specific search terms? I would like to know which BSD it is, how they named their new termios flag which you say does the same thing, and where they inserted it, to get an idea of what to expect if going this route. > Side note: Also the Windows API has a setting for the state of these > lines *after* open (i.e. similar to a termios flag), I would not describe it as being similar to a termios flag, instead I would describe it as similar to TIOCMBIS and TIOCMBIC. Perhaps they retain the state on close-then-reopen sequences - I don't know that part. > and there are > reports of Windows users not expecting the lines to be raised on first > open I read reports from Python users who were migrating apps from Windows to Linux, expecting easy portability because it's Python, and they got bitten by automatic DTR & RTS assertion under Linux which didn't happen for them under Windows. I don't work with Python at all, so I don't know much in the way of details, but supposedly Python maintainers responded that it is a known Linux limitation, that they can control the state of lines only after initial open, and can't avoid an initial glitch - whereas under Windows they did avoid that initial glitch, going through layers of Python, not even using Win32 API directly. > (and behaviour changing between OS releases). For FTDI devices > there appears to be some driver-specific out-of-band mechanism in the > system properties for setting the default behaviour. I didn't know this part. Mychaela, she/her/hers
On Fri, Dec 18, 2020 at 10:03:59AM -0800, Mychaela Falconia wrote: > Greg K-H wrote: > > > We see devices that are "obviously" not the real vid/pid all the time in > > the wild. There's nothing "illegal" about another company using your > > vid/pid, look at all of the ones out there already that use the FTDI > > vendor id yet are "clones", same with pl2303 devices. > > But are those reusers of someone else's VID or PID coming to Linux > kernel maintainers with requests to modify ftdi_sio or pl2303 drivers > to work with their clones? Do you ever see LKML posts along the lines > of "Hi, I am so and so from such and such company, we are not FTDI but > we reuse FTDI's VID and PIDs, but our clone chip does not match the > original and we need to modify the ftdi_sio driver to work with our > poor man's clone chip" - do reusers of someone else's VID or PID come > here with such requests? Users of devices with cloned ids come to us asking why their devices do not work on Linux and how to fix them. Happens all the time, and as the job of a kernel is to enable hardware, we work to make this happen. The vendors who do this are no where to be found, of course, and telling people that the device they just purchased is "counterfit" doesn't really fall in our job description. We have all sorts of work-arounds in Linux drivers to support "fake" devices, our job is to make Linux work for everyone. Anyway, this is far off-topic for this thread, sorry, just letting you know why trying to rely on vid/pid is not the "final solution" people might think it is. I wish someone else would have weighed in on the different api options here, oh well, let me think about this over the next week or so... thanks, greg k-h
Greg K-H wrote: > our job is to make Linux work for everyone. But as your refusal to accept the purely additive (zero impact on anything other than specific hw in question) patch adding support for a new hardware device clearly indicates, your job is NOT to make Linux work for everyone, but rather for a smaller subset of "everyone" *other than* hardware designers who come to the maintainers in good faith, asking to mainline support for new hardware they just made. Johan Hovold wrote: > I couldn't find any such guarantee about the state of these pins when in > output mode in the document you refer to, but that's besides the point. The FTDI document in question shows when the pins in question transition between actively driving outputs and tristate, making it clear to hw engineers that the Hi-Z state must be handled gracefully, practically meaning that pull-up resistors to the I/O voltage rail need to the added. But as one can trivially confirm with an oscilloscope, the power-up transition is from tristate to driving HIGH, not low - the *only* time when the chip will drive low on these outputs is when the USB host explicitly commands it to, either by turning on DTR/RTS in UART mode or by switching to one of the non-UART modes in which these pins acquire different functions. > First, there are other serial devices than those from FTDI. The same principle applies to the initial power-up state of every other reasonable UART chip on the planet. Take the very original 8250 with good old +5V supply voltage - as you first apply power to the chip, the DTR and RTS outputs will be TTL high (corresponding to RS-232 inactive), and they *will not* drive low until and unless you write into the Modem Control Register to change their state. > Second, > these lines can be in any state when the OS boots (e.g. set by a > previous user or OS). If the user is specifically working with a special hardware device that does not tolerate DTR/RTS being randomly jerked around, it is that user's responsibility to ensure that there is NO "previous user or OS" anywhere in the picture. Right now the only active use case (active use case one meaning one with at least one live human actively campaigning for it in the present time) is my DUART28C, and in this active use case the USB-serial interface and the DTR/RTS-sensitive hardware behind this interface are inseparably integrated (a single PCB) into one special USB device. This special USB device does not exist at all to the host computer until the user plugs it in with the intent of operating on it - so there is *no* "previous user or OS" to talk about. > In fact, we already do provide such a way and perhaps that is what we > should use here. We can simply have the ftdi_sio driver not bind to > your control interface, which isn't a serial interface to begin with > anyway. > > Then you're free to use it in whatever way you prefer using libusb. Not acceptable: even though DTR and RTS lines from FT2232D Channel B are repurposed for a non-serial function, the main TxD & RxD lines on that FT2232D channel do form a standard serial interface, and it needs to work as such. Your proposal to use libusb for all serial communication on the secondary channel is unacceptable because it would require writing two versions of every userspace program that needs to talk to the second UART of the Calypso GSM chip: one special libusb-based version for use with DUART28C, and the other standard /dev/ttyXXX version for every other possible serial port to which the second UART of a Calypso GSM chip can be connected, of which there is a great multitude. One part that might not be obvious and probably needs clarification is that the second UART of the Calypso GSM chip (made by TI, not me) is a data-leads-only (TxD & RxD only) interface without any modem control or even hw flow control capability, hence whenever it is connected to any standard UART, that standard UART's modem control and flow control signals are left unconnected - thus DTR and RTS outputs go nowhere. "Standard" userspace software that talks to this UART interface (if one can use the word "standard" for software that is specifically written to speak the proprietary serial protocol of one particular chip vendor) thus does absolutely nothing regarding DTR and RTS - does not touch them in any way, beyond the kernel's automatic assertion on open and negation on close. My only innovation in the DUART28C adapter is to take these two otherwise unused outputs and repurpose them for a different useful function. My userspace programs take -Pdtr and -Prts options that cause these outputs to be pulsed; in the absence of these options (when the very same userspace program is used to talk to the very same TI chip going through a different hw path than my DUART28C), nothing special is done with DTR or RTS, and the kernel's automatic diddling of these lines poses no problem when they are unused and unconnected, as happens with every standard serial port connected to a data-leads- only UART on target hw. But in any case, I give up now. You have made it quite obvious that there is nothing I can say or do to convince you to accept my patch adding support for the DUART28C device, thus every user of this hardware will have to keep applying the needed patch locally on their own system for the rest of eternity. The present DUART28 hardware is not an end user product, instead it is an accessory supplied along with my GSM development boards. The whole setup involves multiple boards interconnected with ribbon cables, and requires an ESD workstation (grounded mat and wrist strap) to use safely, without destroying the hardware with ESD. In other words, not something that can be sold or given out to consumer-level people. However, I am considering making an integrated USB-plug-in GSM modem product, a gadget enclosed in a clear plastic box (hence no ESD control measures needed from the user) with just a USB male-A connector sticking out. Internally the device will consist of the USB-side circuit from DUART28 (FT2232D-based) connected directly to my Calypso modem for GSM - Calypso has no native USB, just two UARTs. This proposed device will be 100% compatible with current DUART28C and will thus share the same custom USB ID - but unlike my current development boards where everything is mostly modular, the boot control interface of DUART28C will be absolutely required to operate the new fully integrated modem gadget. Hence we are going to have an *end user* (as opposed to deep developers only) product that will absolutely require my ftdi_sio patch in order to operate. My earlier references to how Windows handles DTR and RTS were examples for comparison - I am not a Windows programmer (the closest I come is porting some programs from Windows to Linux, including programs that talk via serial ports), I don't have a Windows version of any of my programs for working with TI Calypso GSM gear, thus if I do produce the end user gadget I am thinking about, it will be a product specifically intended for Linux users. And because you are refusing to mainline my patch adding support for the hw in question, every user will be expected to patch their kernel locally, regardless of how new or old their kernel happens to be. (If you were more reasonable, then only old kernel users would need to apply the patch locally, whereas new kernels from some point onward would just work out of the box - but it is not to be, given your stance.) Thus I am going to produce an end user product (a USB-plug-in GSM modem similar to ubiquitous "data sticks", but with the special distinction of fully published source code for GSM radio firmware) specifically for Linux users, accompanied by freely-licensed published-source userspace application software, rather than for users of proprietary OSes like Windows where proprietary drivers and applications are the expected norm. But because you are refusing to accept the hardware support patch which I submitted in good faith and spent enormous amounts of mental and emotional energy defending and justifying, I will have no option but to include a printed leaflet with every piece of hardware explaining the following to the end users: 1) How to apply the needed ftdi_sio driver patch locally; 2) Links to these mailing list threads, showing that we as the hardware vendor have really tried to do everything reasonable to work with the mainline; 3) A suggestion for end users that they should direct their anger and frustration at the specific maintainers who are refusing to accept the simple new hw support patch. Who knows, if you suddenly get hundreds of angry emails from end users (not from me!) who are subjected to the pain of applying a local patch solely because of your obstinence, perhaps you might budge then. > https://reviews.freebsd.org/D20031 I admit that I don't know much about FreeBSD, but the following parts from the above page caught my attention: : This patch allows "stty -f /dev/ttyu9.init -rtsdtr" to prevent : RTS and DTR from being asserted on open(), allowing these devices : to be used without problems. I draw your attention to the /dev/ttyu9.init device name - what exactly does the .init part signify? : .Pp : If : .Dv CNO_RTSDTR : is set then the RTS and DTR lines will not be asserted when the device : is opened. : As a result, this flag is only useful on initial-state devices. Does anyone know what is this "initial-state device" they are talking about? Is it perhaps a special logical device that only manipulates software data structures in the kernel but does not touch the actual hardware, such that opening the .init device for the stty operation will *not* in itself cause the hardware lines to be jerked? If it is so, then FreeBSD has a much more fundamental feature (these "init state" devices) which Linux lacks, and in this case my hardware should indeed work with FreeBSD. *If* the "initial state" devices in FreeBSD really do have the special quality of only manipulating software (kernel) state without touching the actual hardware, *then* despite the superficial similarity in appearance (both using a termios flag and stty), what FreeBSD has done is fundamentally different from what Johan has been proposing with his version of termios flag idea. Johan's idea is that it is perfectly OK to jerk DTR and RTS control lines once for the stty operation. I vehemently disagree with this idea. In many devices one of the lines can be wired as a reset - and let's say your setup is part of a production environment where resetting a live running system is not acceptable. There is also the oft-cited example of radios where DTR and/or RTS turns on the transmitter - now suppose that radio is operating on some tightly regulated frequency related to mission-critical services, and spurious out-of-protocol transmissions are a big no-no. In short, Johan's idea of it being OK to jerk the control signals "just once" is NOT OK with me and never will be, and I will continue to fight against this idea till my last breath. Sincerely, Mychaela, she/her/hers
On Sat, Jan 30, 2021 at 04:18:04PM -0800, Mychaela Falconia wrote: > Greg K-H wrote: > > > our job is to make Linux work for everyone. > > But as your refusal to accept the purely additive (zero impact on > anything other than specific hw in question) patch adding support for > a new hardware device clearly indicates, your job is NOT to make Linux > work for everyone, but rather for a smaller subset of "everyone" > *other than* hardware designers who come to the maintainers in good > faith, asking to mainline support for new hardware they just made. Anything we take adds work to our overall effort to support that new feature added. And you are asking us to do that work for you, for free, for forever. Sorry, given that your attitude does not understand that this is a community and we need to work together, I don't think it's worth continuing here, sorry. If you change your mind in the future, you know how to contact us. greg k-h