Message ID | 20210814025003.2449143-11-colin.foster@in-advantage.com (mailing list archive) |
---|---|
State | RFC |
Delegated to: | Netdev Maintainers |
Headers | show |
Series | add support for VSC75XX control over SPI | expand |
Context | Check | Description |
---|---|---|
netdev/apply | fail | Patch does not apply to net-next |
netdev/tree_selection | success | Clearly marked for net-next |
On Fri, Aug 13, 2021 at 07:50:03PM -0700, Colin Foster wrote: > +* phy_mode = "sgmii": on ports 0, 1, 2, 3 > + port@0 { > + reg = <0>; > + ethernet = <&mac>; > + phy-mode = "sgmii"; > + > + fixed-link { > + speed = <100>; > + full-duplex; > + }; > + }; Your driver is unconditionally setting up the NPI port at gigabit and you claim it works, yet the device tree sees a 100Mbps fixed-link? Which one is right, what speed does the port operate at?
On Sat, Aug 14, 2021 at 02:47:21PM +0300, Vladimir Oltean wrote: > On Fri, Aug 13, 2021 at 07:50:03PM -0700, Colin Foster wrote: > > +* phy_mode = "sgmii": on ports 0, 1, 2, 3 > > > + port@0 { > > + reg = <0>; > > + ethernet = <&mac>; > > + phy-mode = "sgmii"; > > + > > + fixed-link { > > + speed = <100>; > > + full-duplex; > > + }; > > + }; > > Your driver is unconditionally setting up the NPI port at gigabit and > you claim it works, yet the device tree sees a 100Mbps fixed-link? Which > one is right, what speed does the port operate at? Good catch! I made the change to ocelot_spi_vsc7512 yesterday to set it up as gigabit, tested it, and it still works. Previously for my testing I'd had it hard-coded to 100, because the Beaglebone I'm using only supports 100Mbps on eth0. # phytool print swp1/0 ieee-phy: id:0x00070540 ieee-phy: reg:BMCR(0x00) val:0x1040 flags: -reset -loopback [1m+aneg-enable[0m -power-down -isolate -aneg-restart -collision-test speed: 1000-half ieee-phy: reg:BMSR(0x01) val:0x796d capabilities: -100-b4 [1m+100-f[0m [1m+100-h[0m [1m+10-f[0m [1m+10-h[0m -100-t2-f -100-t2-h flags: [1m+ext-status[0m [1m+aneg-complete[0m -remote-fault [1m+aneg-capable[0m [1m+link[0m -jabber [1m+ext-register[0m Of course I understand that "it works" is not the same as "it's correct" What I wanted to accomplish was to use the speed parameter and set up the link based on that. I looked through all the DSA drivers and couldn't find anything that seems to do that. The closest thing I saw was in mt7531_cpu_port_config where they set the speed to either 2500 or 1000 based on the interface. But nothing that I saw would explicitly set the speed based on this parameter. So I think there's something I'm missing. The fixed-link speed should apply to the CPU port on the switch (port@0)? Then eth0 can be manually set to a specific speed, but if it doesn't match the fixed-link speed I'd be out of luck? Or should an ip link or ethtool command to eth0 modify the speeds of both sides of the connection? It feels like setting port@0 to the fastest speed and letting it negotiate down to eth0 makes sense... To ask the same question a different way: I can currently run "ethtool -s eth0 speed 10 duplex full autoneg on" and the link at eth0 drops to 10Mbps. Pinging my desktop jumps from about 400us to about 600us when I do that. Should I not be able to do that? It should be fixed at 100Mbps without autoneg, end of story? Because in the current configuration it feels like the fixed-link settings are more a suggestion than a rule...
On Sat, Aug 14, 2021 at 11:40:40AM -0700, Colin Foster wrote: > On Sat, Aug 14, 2021 at 02:47:21PM +0300, Vladimir Oltean wrote: > > On Fri, Aug 13, 2021 at 07:50:03PM -0700, Colin Foster wrote: > > > +* phy_mode = "sgmii": on ports 0, 1, 2, 3 > > > > > + port@0 { > > > + reg = <0>; > > > + ethernet = <&mac>; > > > + phy-mode = "sgmii"; > > > + > > > + fixed-link { > > > + speed = <100>; > > > + full-duplex; > > > + }; > > > + }; > > > > Your driver is unconditionally setting up the NPI port at gigabit and > > you claim it works, yet the device tree sees a 100Mbps fixed-link? Which > > one is right, what speed does the port operate at? > > Good catch! > > I made the change to ocelot_spi_vsc7512 yesterday to set it up as > gigabit, tested it, and it still works. Previously for my testing I'd > had it hard-coded to 100, because the Beaglebone I'm using only supports > 100Mbps on eth0. > > # phytool print swp1/0 Why are you showing the PHY registers of swp1? Why are these relevant at all? > > ieee-phy: id:0x00070540 > > ieee-phy: reg:BMCR(0x00) val:0x1040 > flags: -reset -loopback [1m+aneg-enable[0m -power-down -isolate -aneg-restart -collision-test > speed: 1000-half Also, 1000/half sounds like an odd speed to end negotiation at. > > ieee-phy: reg:BMSR(0x01) val:0x796d > capabilities: -100-b4 [1m+100-f[0m [1m+100-h[0m [1m+10-f[0m [1m+10-h[0m -100-t2-f -100-t2-h > flags: [1m+ext-status[0m [1m+aneg-complete[0m -remote-fault [1m+aneg-capable[0m [1m+link[0m -jabber [1m+ext-register[0m > > > Of course I understand that "it works" is not the same as "it's correct" > > What I wanted to accomplish was to use the speed parameter and set up > the link based on that. I looked through all the DSA drivers and > couldn't find anything that seems to do that. The closest thing I saw > was in mt7531_cpu_port_config where they set the speed to either 2500 or > 1000 based on the interface. But nothing that I saw would explicitly set > the speed based on this parameter. As I mentioned in the other email, .phylink_mac_link_up is the function you are looking for. Phylink parses the fixed-link and calls that function for fixed-link ports with the speed and duplex specified. Check and see if felix_phylink_mac_link_up is not in fact called with link_an_mode == MLO_AN_FIXED, speed == SPEED_100 and duplex == DUPLEX_FULL, then what you are doing with that and if it makes sense for what you are trying to do. > > So I think there's something I'm missing. The fixed-link speed should apply to > the CPU port on the switch (port@0)? Is this a question? It is under port@0, the port with the 'ethernet' property i.e. the CPU port, so why should it not? > Then eth0 can be manually set to a specific speed, but if it doesn't > match the fixed-link speed I'd be out of luck? Or should an ip link or > ethtool command to eth0 modify the speeds of both sides of the > connection? It feels like setting port@0 to the fastest speed and > letting it negotiate down to eth0 makes sense... > > To ask the same question a different way: > > I can currently run "ethtool -s eth0 speed 10 duplex full autoneg on" > and the link at eth0 drops to 10Mbps. Pinging my desktop jumps from > about 400us to about 600us when I do that. If eth0 is also a fixed-link, you should not be able to do that, no. But the fact that you are able to do that means it's not a fixed-link, you have a pair of PHYs that freely auto-negotiate the speed between the BeagleBone and the switch. > > Should I not be able to do that? It should be fixed at 100Mbps without > autoneg, end of story? Because in the current configuration it feels > like the fixed-link settings are more a suggestion than a rule... > It should describe the hardware configuration, of course. It is incorrect to describe one side of a copper PHY connection as fixed-link and the other as having a phy-handle, and it sounds like this is what you're doing. We need to see the device tree binding for eth0, and maybe a picture of your setup if that is possible. How do you connect the switch board to the BeagleBone? Is it an RJ45 cable or some sort of PCIe-style connector with fingers for an SGMII SERDES lane, in which the board is plugged? The device tree says SGMII, the behavior says RJ45.
On Sat, Aug 14, 2021 at 10:08:54PM +0300, Vladimir Oltean wrote: > On Sat, Aug 14, 2021 at 11:40:40AM -0700, Colin Foster wrote: > > On Sat, Aug 14, 2021 at 02:47:21PM +0300, Vladimir Oltean wrote: > > > On Fri, Aug 13, 2021 at 07:50:03PM -0700, Colin Foster wrote: > > > > +* phy_mode = "sgmii": on ports 0, 1, 2, 3 > > > > > > > + port@0 { > > > > + reg = <0>; > > > > + ethernet = <&mac>; > > > > + phy-mode = "sgmii"; > > > > + > > > > + fixed-link { > > > > + speed = <100>; > > > > + full-duplex; > > > > + }; > > > > + }; > > > > > > Your driver is unconditionally setting up the NPI port at gigabit and > > > you claim it works, yet the device tree sees a 100Mbps fixed-link? Which > > > one is right, what speed does the port operate at? > > > > Good catch! > > > > I made the change to ocelot_spi_vsc7512 yesterday to set it up as > > gigabit, tested it, and it still works. Previously for my testing I'd > > had it hard-coded to 100, because the Beaglebone I'm using only supports > > 100Mbps on eth0. > > > > # phytool print swp1/0 > > Why are you showing the PHY registers of swp1? Why are these relevant at all? Some sleight of hand. I'm not sure there's a different way to do it, but running print on swp1/0 gives phy address 0 on the bus that contains swp1. So in this setup, swp1/0 is the same as swp2/0, swp3/0 and swp4/0, all of which point to the registers associated with "swp0" from what I understand. > > > > > ieee-phy: id:0x00070540 > > > > ieee-phy: reg:BMCR(0x00) val:0x1040 > > flags: -reset -loopback [1m+aneg-enable[0m -power-down -isolate -aneg-restart -collision-test > > speed: 1000-half > > Also, 1000/half sounds like an odd speed to end negotiation at. Agreed. Possibly a misunderstanding by me during vsc7512_enable_npi_port? I'll look into this. > > > > > ieee-phy: reg:BMSR(0x01) val:0x796d > > capabilities: -100-b4 [1m+100-f[0m [1m+100-h[0m [1m+10-f[0m [1m+10-h[0m -100-t2-f -100-t2-h > > flags: [1m+ext-status[0m [1m+aneg-complete[0m -remote-fault [1m+aneg-capable[0m [1m+link[0m -jabber [1m+ext-register[0m > > > > > > Of course I understand that "it works" is not the same as "it's correct" > > > > What I wanted to accomplish was to use the speed parameter and set up > > the link based on that. I looked through all the DSA drivers and > > couldn't find anything that seems to do that. The closest thing I saw > > was in mt7531_cpu_port_config where they set the speed to either 2500 or > > 1000 based on the interface. But nothing that I saw would explicitly set > > the speed based on this parameter. > > As I mentioned in the other email, .phylink_mac_link_up is the function > you are looking for. Phylink parses the fixed-link and calls that > function for fixed-link ports with the speed and duplex specified. Check > and see if felix_phylink_mac_link_up is not in fact called with > link_an_mode == MLO_AN_FIXED, speed == SPEED_100 and duplex == DUPLEX_FULL, > then what you are doing with that and if it makes sense for what you are > trying to do. I'll reply there once I've absorbed everything. > > > > > So I think there's something I'm missing. The fixed-link speed should apply to > > the CPU port on the switch (port@0)? > > Is this a question? It is under port@0, the port with the 'ethernet' > property i.e. the CPU port, so why should it not? > > > Then eth0 can be manually set to a specific speed, but if it doesn't > > match the fixed-link speed I'd be out of luck? Or should an ip link or > > ethtool command to eth0 modify the speeds of both sides of the > > connection? It feels like setting port@0 to the fastest speed and > > letting it negotiate down to eth0 makes sense... > > > > To ask the same question a different way: > > > > I can currently run "ethtool -s eth0 speed 10 duplex full autoneg on" > > and the link at eth0 drops to 10Mbps. Pinging my desktop jumps from > > about 400us to about 600us when I do that. > > If eth0 is also a fixed-link, you should not be able to do that, no. > But the fact that you are able to do that means it's not a fixed-link, > you have a pair of PHYs that freely auto-negotiate the speed between the > BeagleBone and the switch. Yes, that is my setup. Addressed below. > > > > > Should I not be able to do that? It should be fixed at 100Mbps without > > autoneg, end of story? Because in the current configuration it feels > > like the fixed-link settings are more a suggestion than a rule... > > > > It should describe the hardware configuration, of course. It is > incorrect to describe one side of a copper PHY connection as fixed-link > and the other as having a phy-handle, and it sounds like this is what > you're doing. We need to see the device tree binding for eth0, and > maybe a picture of your setup if that is possible. How do you connect > the switch board to the BeagleBone? Is it an RJ45 cable or some sort of > PCIe-style connector with fingers for an SGMII SERDES lane, in which the > board is plugged? > > The device tree says SGMII, the behavior says RJ45. I'm using the standard BeagleBone devicetree, so &mac is defined in arch/arm/boot/dts/am335x-bone-common.dtsi. For this stage of development I'm using an ethernet cable plugged from the BeagleBone to port 0 of the VSC7512 dev board. I haven't done anything more to the am335x-boneblack.dts other than add the spi and switch configurations. The connections between the beaglebone and dev board are limited to the 4 SPI lines, a ground, and an ethernet cable. So DSA requires a fixed-link property. And that makes sense... who in their right mind would connect switches on a board using an RJ45 connection :) Then the only reason any of this is working is because I have eth0 set up as an RJ45 connection, and because of that I need the hack to enable the phy on the switch port 0... Maybe that's a question: Is my devicetree incorrect for claiming the connection is SGMII when it should be RJ45? Or is my setup incorrect for using RJ45 and there's no way to configure it that way, so the fact that it functions is an anomaly?
On Sat, Aug 14, 2021 at 04:41:58PM -0700, Colin Foster wrote: > So DSA requires a fixed-link property. How did you come to that conclusion? As mentioned twice already, DSA registers a phylink for the CPU port, and phylink works with either a phy-handle or a fixed-link. Support for this has been added more than 2 years ago: https://patchwork.ozlabs.org/project/netdev/patch/1558992127-26008-11-git-send-email-ioana.ciornei@nxp.com/ You have a PHY... so use a phy-handle. > And that makes sense... who in > their right mind would connect switches on a board using an RJ45 > connection :) Then the only reason any of this is working is because I > have eth0 set up as an RJ45 connection, and because of that I need the > hack to enable the phy on the switch port 0... > > Maybe that's a question: > Is my devicetree incorrect for claiming the connection is SGMII when it > should be RJ45? Your device tree description is absolutely incorrect by all accounts. First of all, "is SGMII" does not really preclude "is RJ45", because you can have an external PHY connected to your MAC via SGMII, and that external PHY would provide RJ45 access. That would be absolutely fine too. That would be described as: port@0 { phy-mode = "sgmii"; phy-handle = <&external_phy>; }; It would be absolutely fine as well to describe the RJ45 port via an internal PHY if that's how things are hooked up in your eval board (really don't know what PHY you have, sorry): port@0 { phy-mode = "internal"; phy-handle = <&internal_phy>; }; But in the absence of a phy-handle and the presence of fixed-link, like the way you are describing it, you are telling Linux that you have an SGMII PHY-less system, where the SGMII lane goes directly towards the outside world. I think it is actually written somewhere in the documentation that describing a connection to a PHY using a fixed-link is wrong and strongly discouraged. > Or is my setup incorrect for using RJ45 and there's no > way to configure it that way, so the fact that it functions is an > anomaly? No, the setup is not incorrect, it is just fine and both DSA and phylink support it as long as it is described properly, with the adequate phy-handle on the CPU port.
On Sun, Aug 15, 2021 at 03:00:41AM +0300, Vladimir Oltean wrote: > On Sat, Aug 14, 2021 at 04:41:58PM -0700, Colin Foster wrote: > > So DSA requires a fixed-link property. > > How did you come to that conclusion? As mentioned twice already, DSA > registers a phylink for the CPU port, and phylink works with either a > phy-handle or a fixed-link. > > Support for this has been added more than 2 years ago: > https://patchwork.ozlabs.org/project/netdev/patch/1558992127-26008-11-git-send-email-ioana.ciornei@nxp.com/ > > You have a PHY... so use a phy-handle. My misunderstanding. I think I saw all the documentation / examples and understood that to be "it must be this way". I shouldn't have drawn that conclusion. > > > And that makes sense... who in > > their right mind would connect switches on a board using an RJ45 > > connection :) Then the only reason any of this is working is because I > > have eth0 set up as an RJ45 connection, and because of that I need the > > hack to enable the phy on the switch port 0... > > > > Maybe that's a question: > > Is my devicetree incorrect for claiming the connection is SGMII when it > > should be RJ45? > > Your device tree description is absolutely incorrect by all accounts. > > First of all, "is SGMII" does not really preclude "is RJ45", because you > can have an external PHY connected to your MAC via SGMII, and that > external PHY would provide RJ45 access. That would be absolutely fine too. > > That would be described as: > > port@0 { > phy-mode = "sgmii"; > phy-handle = <&external_phy>; > }; > > It would be absolutely fine as well to describe the RJ45 port via an > internal PHY if that's how things are hooked up in your eval board > (really don't know what PHY you have, sorry): > > port@0 { > phy-mode = "internal"; > phy-handle = <&internal_phy>; > }; > > But in the absence of a phy-handle and the presence of fixed-link, like > the way you are describing it, you are telling Linux that you have an > SGMII PHY-less system, where the SGMII lane goes directly towards the > outside world. > Understood, and thank you for the feedback. I am definitely not currently running in a PHY-less system on this interface. I also had some confusion about phy-mode = "internal" vs phy-mode = "sgmii". It seems like I have ports 1-3 incorrectly confiugred as well - they are internal to the VSC7512 chip. > I think it is actually written somewhere in the documentation that > describing a connection to a PHY using a fixed-link is wrong and > strongly discouraged. I have some reading to do. I made assumptions early on and now that things seem to be getting close, it is becoming clear that those misunderstandings were leading me down the wrong path. > > > Or is my setup incorrect for using RJ45 and there's no > > way to configure it that way, so the fact that it functions is an > > anomaly? > > No, the setup is not incorrect, it is just fine and both DSA and phylink > support it as long as it is described properly, with the adequate > phy-handle on the CPU port. This is very good to know. Thank you. I'm sorry that you're having to troubleshoot my devicetree, but it is incerdibly helpful to just know "the devicetree is wrong" instead of "my implementation of the driver has this shortcoming." I have another round of dev / testing ahead of me. Again, a sincere thank you for the feedback. I understand I'm making a lot of mistakes. I'm hopeful that these mistakes can come off as "inexperienced" and, frankly, that I'm not annoying everyone.
On Fri, Aug 13, 2021 at 07:50:03PM -0700, Colin Foster wrote: > Signed-off-by: Colin Foster <colin.foster@in-advantage.com> > --- > .../devicetree/bindings/net/dsa/ocelot.txt | 92 +++++++++++++++++++ > 1 file changed, 92 insertions(+) > > diff --git a/Documentation/devicetree/bindings/net/dsa/ocelot.txt b/Documentation/devicetree/bindings/net/dsa/ocelot.txt > index 7a271d070b72..edf560a50803 100644 > --- a/Documentation/devicetree/bindings/net/dsa/ocelot.txt > +++ b/Documentation/devicetree/bindings/net/dsa/ocelot.txt > @@ -8,6 +8,7 @@ Currently the switches supported by the felix driver are: > > - VSC9959 (Felix) > - VSC9953 (Seville) > +- VSC7511, VSC7512, VSC7513, VSC7514 via SPI > > The VSC9959 switch is found in the NXP LS1028A. It is a PCI device, part of the > larger ENETC root complex. As a result, the ethernet-switch node is a sub-node > @@ -211,3 +212,94 @@ Example: > }; > }; > }; > + > +The VSC7513 and VSC7514 switches can be controlled internally via the MIPS > +processor. The VSC7511 and VSC7512 don't have this internal processor, but all > +four chips can be controlled externally through SPI with the following required > +properties: > + > +- compatible: > + Can be "mscc,vsc7511", "mscc,vsc7512", "mscc,vsc7513", or > + "mscc,vsc7514". > + > +Supported phy modes for all chips are: > + > +* phy_mode = "sgmii": on ports 0, 1, 2, 3 > + > +The VSC7512 and 7514 also support: > + > +* phy_mode = "sgmii": on ports 4, 5, 6, 7 > +* phy_mode = "qsgmii": on ports 7, 8, 10 > + > +Example for control from a BeagleBone Black > + > +&spi0 { > + #address-cells = <1>; > + #size-cells = <0>; > + > + ethernet-switch@0 { > + compatible = "mscc,vsc7512"; > + spi-max-frequency = <250000>; > + reg = <0>; > + > + ports { > + #address-cells = <1>; > + #size-cells = <0>; > + > + port@0 { > + reg = <0>; > + ethernet = <&mac>; > + phy-mode = "sgmii"; > + > + fixed-link { > + speed = <100>; > + full-duplex; > + }; > + }; > + > + port@1 { > + reg = <1>; > + label = "swp1"; > + phy-handle = <&sw_phy1>; > + phy-mode = "sgmii"; > + }; > + > + port@2 { > + reg = <2>; > + label = "swp2"; > + phy-handle = <&sw_phy2>; > + phy-mode = "sgmii"; > + }; > + > + port@3 { > + reg = <3>; > + label = "swp3"; > + phy-handle = <&sw_phy3>; > + phy-mode = "sgmii"; > + }; > + }; > + > + mdio { > + #address-cells = <1>; > + #size-cells = <0>; > + > + sw_phy1: ethernet-phy@1 { > + #address-cells = <1>; > + #size-cells = <0>; > + reg = <0x1>; > + }; > + > + sw_phy2: ethernet-phy@2 { > + #address-cells = <1>; > + #size-cells = <0>; > + reg = <0x2>; > + }; > + > + sw_phy3: ethernet-phy@3 { > + #address-cells = <1>; > + #size-cells = <0>; > + reg = <0x3>; > + }; > + }; > + }; > +}; If you want a whole new example, then convert this to DT schema. But is there anything really new or different here to warrant another example? Rob
diff --git a/Documentation/devicetree/bindings/net/dsa/ocelot.txt b/Documentation/devicetree/bindings/net/dsa/ocelot.txt index 7a271d070b72..edf560a50803 100644 --- a/Documentation/devicetree/bindings/net/dsa/ocelot.txt +++ b/Documentation/devicetree/bindings/net/dsa/ocelot.txt @@ -8,6 +8,7 @@ Currently the switches supported by the felix driver are: - VSC9959 (Felix) - VSC9953 (Seville) +- VSC7511, VSC7512, VSC7513, VSC7514 via SPI The VSC9959 switch is found in the NXP LS1028A. It is a PCI device, part of the larger ENETC root complex. As a result, the ethernet-switch node is a sub-node @@ -211,3 +212,94 @@ Example: }; }; }; + +The VSC7513 and VSC7514 switches can be controlled internally via the MIPS +processor. The VSC7511 and VSC7512 don't have this internal processor, but all +four chips can be controlled externally through SPI with the following required +properties: + +- compatible: + Can be "mscc,vsc7511", "mscc,vsc7512", "mscc,vsc7513", or + "mscc,vsc7514". + +Supported phy modes for all chips are: + +* phy_mode = "sgmii": on ports 0, 1, 2, 3 + +The VSC7512 and 7514 also support: + +* phy_mode = "sgmii": on ports 4, 5, 6, 7 +* phy_mode = "qsgmii": on ports 7, 8, 10 + +Example for control from a BeagleBone Black + +&spi0 { + #address-cells = <1>; + #size-cells = <0>; + + ethernet-switch@0 { + compatible = "mscc,vsc7512"; + spi-max-frequency = <250000>; + reg = <0>; + + ports { + #address-cells = <1>; + #size-cells = <0>; + + port@0 { + reg = <0>; + ethernet = <&mac>; + phy-mode = "sgmii"; + + fixed-link { + speed = <100>; + full-duplex; + }; + }; + + port@1 { + reg = <1>; + label = "swp1"; + phy-handle = <&sw_phy1>; + phy-mode = "sgmii"; + }; + + port@2 { + reg = <2>; + label = "swp2"; + phy-handle = <&sw_phy2>; + phy-mode = "sgmii"; + }; + + port@3 { + reg = <3>; + label = "swp3"; + phy-handle = <&sw_phy3>; + phy-mode = "sgmii"; + }; + }; + + mdio { + #address-cells = <1>; + #size-cells = <0>; + + sw_phy1: ethernet-phy@1 { + #address-cells = <1>; + #size-cells = <0>; + reg = <0x1>; + }; + + sw_phy2: ethernet-phy@2 { + #address-cells = <1>; + #size-cells = <0>; + reg = <0x2>; + }; + + sw_phy3: ethernet-phy@3 { + #address-cells = <1>; + #size-cells = <0>; + reg = <0x3>; + }; + }; + }; +};
Signed-off-by: Colin Foster <colin.foster@in-advantage.com> --- .../devicetree/bindings/net/dsa/ocelot.txt | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+)