| Message ID | 54F82ED1.8030900@plexistor.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Mar 5, 2015 at 2:24 AM, Boaz Harrosh <boaz@plexistor.com> wrote: > > There are multiple vendors of DDR3 NvDIMMs out in the market today. > At various stages of development/production. It is estimated that > there are already more the 100ds of thousands chips sold to > testers and sites. > > All the BIOS vendors I know of, tagged these chips at e820 table > as type-12 memory. > > Now the ACPI comity, as far as I know, did not yet define a > standard type for NvDIMM. Also, as far as I know any NvDIMM > standard will only be defined for DDR4. So DDR3 NvDIMM is > probably stuck with this none STD type. There's no relation between E820 types and DDR technology revisions. > I Wish and call the ACPI comity to Define that NvDIMM is type-12. > Also for DDR4 > > The motivation of this patch is to be able to differentiate > this NvDIMM type from a real future "unknown-reserved" type. > > In this patch I name type-12 "unknown-12". This is because of > ACPI politics that refuse to reserve type-12 as DDR3-NvDIMM It's not "politics". Setting standards takes time and the platforms in question simply jumped the gun to enable a proof-of-concept. > and members keep saying: > "What if ACPI assigns type-12 for something else in future" > > [And I say: Then just don't. Please?] Once a standard number is assigned, platform firmwares can update type-12 to that number. We might consider a compile time override for these niche/pre-standard systems that can't/won't update, but it's not clear to me that we even need to go that far.
On Thu, Mar 5, 2015 at 12:56 PM, Dan Williams <dan.j.williams@intel.com> wrote: > On Thu, Mar 5, 2015 at 2:24 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >> >> There are multiple vendors of DDR3 NvDIMMs out in the market today. >> At various stages of development/production. It is estimated that >> there are already more the 100ds of thousands chips sold to >> testers and sites. >> >> All the BIOS vendors I know of, tagged these chips at e820 table >> as type-12 memory. >> >> Now the ACPI comity, as far as I know, did not yet define a >> standard type for NvDIMM. Also, as far as I know any NvDIMM >> standard will only be defined for DDR4. So DDR3 NvDIMM is >> probably stuck with this none STD type. > > There's no relation between E820 types and DDR technology revisions. > >> I Wish and call the ACPI comity to Define that NvDIMM is type-12. >> Also for DDR4 >> >> The motivation of this patch is to be able to differentiate >> this NvDIMM type from a real future "unknown-reserved" type. >> >> In this patch I name type-12 "unknown-12". This is because of >> ACPI politics that refuse to reserve type-12 as DDR3-NvDIMM > > It's not "politics". Setting standards takes time and the platforms > in question simply jumped the gun to enable a proof-of-concept. > >> and members keep saying: >> "What if ACPI assigns type-12 for something else in future" >> >> [And I say: Then just don't. Please?] > > Once a standard number is assigned, platform firmwares can update > type-12 to that number. We might consider a compile time override for > these niche/pre-standard systems that can't/won't update, but it's not > clear to me that we even need to go that far. I will be shocked if a standard of this form ever appears. Modern systems *don't have e820*. The BIOSes that are using this type 12 hack are awful throwbacks. --Andy
On 03/05/2015 10:56 PM, Dan Williams wrote: > On Thu, Mar 5, 2015 at 2:24 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >> <> >> Now the ACPI comity, as far as I know, did not yet define a >> standard type for NvDIMM. Also, as far as I know any NvDIMM >> standard will only be defined for DDR4. So DDR3 NvDIMM is >> probably stuck with this none STD type. > > There's no relation between E820 types and DDR technology revisions. > Yes and no, I mean the DDR4 has extra legs and signals defined for NvDIMM. So DDR3 will always mean different style of NvDIMM. You tell me. Say the standard finally comes out. Will I have a new bios from Intel for my DDR3 system here in the lab that will report the new STD type ? What I meant is that DDR3 is too old for the proposed STD and probably only DDR4 NvDIMMs will be supported in systems. The way the STD defined it. <> >> In this patch I name type-12 "unknown-12". This is because of >> ACPI politics that refuse to reserve type-12 as DDR3-NvDIMM > > It's not "politics". Setting standards takes time and the platforms > in question simply jumped the gun to enable a proof-of-concept. > So ye, but once you have 100,000 devices out there, then the dichotomy between standards-takes-time vs proof-of-concept, becomes politics. This is the definition of politics, when life moves faster than some "body", the "body" stands on its back feet and shoots fire from his head. >> and members keep saying: >> "What if ACPI assigns type-12 for something else in future" >> >> [And I say: Then just don't. Please?] > > Once a standard number is assigned, platform firmwares can update > type-12 to that number. We might consider a compile time override for > these niche/pre-standard systems that can't/won't update, but it's not > clear to me that we even need to go that far. > OK, so I do not understand what you want. Yes or No to this patch? This patch with unknown-12 is for NOW. For systems already running. So we can differentiate between reserved-unknown which might mean type-13 and this here bastard type-12 which we know is NvDIMM but for future sake we do not call by name? Or maybe we should call it NVDIMM-12 ? Thanks Boaz
On 03/06/2015 01:09 AM, Andy Lutomirski wrote: <> > > I will be shocked if a standard of this form ever appears. Modern > systems *don't have e820*. The BIOSes that are using this type 12 > hack are awful throwbacks. So far the systems we have, with DDR4 NvDIMM(s) (Actual chips arriving soon) still have BIOS (On by default). The BIOS was yelled "Wolfe" for so long NOW ;-) Again these are working system in the field, who will switch them all to UEFI? How will the UEFI present them to the system? can you point me to the relevant code? (Or did you mean BIOS but with a different communication path than e820?) > > --Andy > Thanks Boaz
On Mon, Mar 9, 2015 at 7:19 AM, Boaz Harrosh <boaz@plexistor.com> wrote: > On 03/05/2015 10:56 PM, Dan Williams wrote: [..] >> It's not "politics". Setting standards takes time and the platforms >> in question simply jumped the gun to enable a proof-of-concept. >> > > So ye, but once you have 100,000 devices out there, then the dichotomy > between standards-takes-time vs proof-of-concept, becomes politics. Ok. ...although, I have a question about this "100,000 devices" you quote. What's not clear to me is how many platforms are shipping with type-12. Certainly there are very many NVDIMMs on the market, but which off-the-shelf systems can one obtain that include type-12 support? Not that it would change the disposition of this patch, if platforms are in the field they're "in the field!", just clarifying that "100,000 devices" is NVDIMMs not type-12 platforms, right?
On Mon, Mar 9, 2015 at 7:44 AM, Dan Williams <dan.j.williams@intel.com> wrote: > On Mon, Mar 9, 2015 at 7:19 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >> On 03/05/2015 10:56 PM, Dan Williams wrote: > [..] >>> It's not "politics". Setting standards takes time and the platforms >>> in question simply jumped the gun to enable a proof-of-concept. >>> >> >> So ye, but once you have 100,000 devices out there, then the dichotomy >> between standards-takes-time vs proof-of-concept, becomes politics. > > Ok. > > ...although, I have a question about this "100,000 devices" you quote. > What's not clear to me is how many platforms are shipping with > type-12. Certainly there are very many NVDIMMs on the market, but > which off-the-shelf systems can one obtain that include type-12 > support? Not that it would change the disposition of this patch, if > platforms are in the field they're "in the field!", just clarifying > that "100,000 devices" is NVDIMMs not type-12 platforms, right? This is a type-12 platform: http://www.supermicro.com/products/motherboard/Xeon/C600/X9DRH-iF-NV.cfm You can order one online, for example: http://www.acmemicro.com/Product/11926/Supermicro+X9DRH-iF-NV-O+Server+Board+DP+Xeon+E5-2600+LGA2011+DDR3+SATA3+RAID+IPMI+GbE+PCIe+ATX+MBD-X9DRH-iF-NV --Andy
On Mon, Mar 9, 2015 at 11:14 AM, Andy Lutomirski <luto@amacapital.net> wrote: > On Mon, Mar 9, 2015 at 7:44 AM, Dan Williams <dan.j.williams@intel.com> wrote: >> On Mon, Mar 9, 2015 at 7:19 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >>> On 03/05/2015 10:56 PM, Dan Williams wrote: >> [..] >>>> It's not "politics". Setting standards takes time and the platforms >>>> in question simply jumped the gun to enable a proof-of-concept. >>>> >>> >>> So ye, but once you have 100,000 devices out there, then the dichotomy >>> between standards-takes-time vs proof-of-concept, becomes politics. >> >> Ok. >> >> ...although, I have a question about this "100,000 devices" you quote. >> What's not clear to me is how many platforms are shipping with >> type-12. Certainly there are very many NVDIMMs on the market, but >> which off-the-shelf systems can one obtain that include type-12 >> support? Not that it would change the disposition of this patch, if >> platforms are in the field they're "in the field!", just clarifying >> that "100,000 devices" is NVDIMMs not type-12 platforms, right? > > This is a type-12 platform: > > http://www.supermicro.com/products/motherboard/Xeon/C600/X9DRH-iF-NV.cfm > > You can order one online, for example: > > http://www.acmemicro.com/Product/11926/Supermicro+X9DRH-iF-NV-O+Server+Board+DP+Xeon+E5-2600+LGA2011+DDR3+SATA3+RAID+IPMI+GbE+PCIe+ATX+MBD-X9DRH-iF-NV > Ok, the gig is up :)
Hi, On Mon, Mar 09, 2015 at 02:10:37PM +0200, Boaz Harrosh wrote: > On 03/06/2015 01:09 AM, Andy Lutomirski wrote: > <> > > > > I will be shocked if a standard of this form ever appears. Modern > > systems *don't have e820*. The BIOSes that are using this type 12 > > hack are awful throwbacks. > > So far the systems we have, with DDR4 NvDIMM(s) (Actual chips arriving soon) > still have BIOS (On by default). The BIOS was yelled "Wolfe" for so long NOW ;-) > Again these are working system in the field, who will switch them all to UEFI? > > How will the UEFI present them to the system? can you point me to the relevant > code? (Or did you mean BIOS but with a different communication path than e820?) > Per my understand... With EFI Boot Stub, there have setup_e820() codes in arch/x86/boot/compressed/eboot.c that used to transfer EFI memmap to e820 entries. Currently doesn't have any EFI_MEMORY_TYPE reflects to NvDIMM that will map to e820_type. I wonder what kind of EFI_MEMORY_TYPE reported by UEFI BIOS on those "shipped NvDIMMs motherboards", like supermicro X9DRH-iF-NV. Then we may need add code to setup_e820() for mapping the efi memory type to e820 type 12 region. > > > > --Andy > > > > Thanks > Boaz > Thanks a lot! Joey Lee
On 03/09/2015 05:17 PM, Dan Williams wrote: > On Mon, Mar 9, 2015 at 11:14 AM, Andy Lutomirski <luto@amacapital.net> wrote: >> On Mon, Mar 9, 2015 at 7:44 AM, Dan Williams <dan.j.williams@intel.com> wrote: >>> On Mon, Mar 9, 2015 at 7:19 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >>>> On 03/05/2015 10:56 PM, Dan Williams wrote: >>> [..] >>>>> It's not "politics". Setting standards takes time and the platforms >>>>> in question simply jumped the gun to enable a proof-of-concept. >>>>> >>>> >>>> So ye, but once you have 100,000 devices out there, then the dichotomy >>>> between standards-takes-time vs proof-of-concept, becomes politics. >>> >>> Ok. >>> >>> ...although, I have a question about this "100,000 devices" you quote. >>> What's not clear to me is how many platforms are shipping with >>> type-12. Certainly there are very many NVDIMMs on the market, but >>> which off-the-shelf systems can one obtain that include type-12 >>> support? Not that it would change the disposition of this patch, if >>> platforms are in the field they're "in the field!", just clarifying >>> that "100,000 devices" is NVDIMMs not type-12 platforms, right? >> >> This is a type-12 platform: >> >> http://www.supermicro.com/products/motherboard/Xeon/C600/X9DRH-iF-NV.cfm >> >> You can order one online, for example: >> >> http://www.acmemicro.com/Product/11926/Supermicro+X9DRH-iF-NV-O+Server+Board+DP+Xeon+E5-2600+LGA2011+DDR3+SATA3+RAID+IPMI+GbE+PCIe+ATX+MBD-X9DRH-iF-NV >> > > Ok, the gig is up :) > Hi Dan. Sigh, yes. These systems are out there. lots of them, and not only at developers. So far all the system I had a hands on and all the people I've asked all reported type-12, if the DIMM was reported at all (or even booted). With various success rate of actual NVIDIMM functionality. I call to all developers that are working on NvDIMM. What are the methods you see that these chips are presented to the system? Are you using BIOS or UEFI? (From what I understood, there is actually only one family of motherboards/firmware That actually boot NVDIMMs, and it all originated from Intel. I might be off by a mile, I have not signed any NDA and no one told me so, just my personal follow the bread crumbs) Could we please reconsider and name those in /proc/iomem as NVDIMM-12 instead of unknow-12 just as the nobleness obliges? We are tainting the Kernel and complaining at driver use of them, that will not change, just the name? Thanks Boaz
On 03/10/2015 07:11 AM, joeyli wrote: <> > > Per my understand... > > With EFI Boot Stub, there have setup_e820() codes in arch/x86/boot/compressed/eboot.c > that used to transfer EFI memmap to e820 entries. Currently doesn't have any > EFI_MEMORY_TYPE reflects to NvDIMM that will map to e820_type. > > I wonder what kind of EFI_MEMORY_TYPE reported by UEFI BIOS on those "shipped > NvDIMMs motherboards", like supermicro X9DRH-iF-NV. Then we may need add code > to setup_e820() for mapping the efi memory type to e820 type 12 region. > Thanks Joey. This is tremendous help. (Just saved me days of research) Our DDR4 NvDIMM chips arrive this week I will try and debug all this, also booting EFI. If you have any chance to experiments in your lab, and compare notes it would be great. <> > > Thanks a lot! > Joey Lee > Thanks Boaz
On Mar 10, 2015 1:12 AM, "joeyli" <jlee@suse.com> wrote: > > Hi, > > On Mon, Mar 09, 2015 at 02:10:37PM +0200, Boaz Harrosh wrote: > > On 03/06/2015 01:09 AM, Andy Lutomirski wrote: > > <> > > > > > > I will be shocked if a standard of this form ever appears. Modern > > > systems *don't have e820*. The BIOSes that are using this type 12 > > > hack are awful throwbacks. > > > > So far the systems we have, with DDR4 NvDIMM(s) (Actual chips arriving soon) > > still have BIOS (On by default). The BIOS was yelled "Wolfe" for so long NOW ;-) > > Again these are working system in the field, who will switch them all to UEFI? > > > > How will the UEFI present them to the system? can you point me to the relevant > > code? (Or did you mean BIOS but with a different communication path than e820?) > > > > Per my understand... > > With EFI Boot Stub, there have setup_e820() codes in arch/x86/boot/compressed/eboot.c > that used to transfer EFI memmap to e820 entries. Currently doesn't have any > EFI_MEMORY_TYPE reflects to NvDIMM that will map to e820_type. > > I wonder what kind of EFI_MEMORY_TYPE reported by UEFI BIOS on those "shipped > NvDIMMs motherboards", like supermicro X9DRH-iF-NV. Then we may need add code > to setup_e820() for mapping the efi memory type to e820 type 12 region. Nothing useful. You can't detect the NvDIMM in EFI mode on my board as far as I know. Rumor has it that we'll learn more about the EFI case very soon. --Andy > > > > > > > --Andy > > > > > > > Thanks > > Boaz > > > > Thanks a lot! > Joey Lee
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c index c2f2da2..4bf574a 100644 --- a/arch/x86/kernel/e820.c +++ b/arch/x86/kernel/e820.c @@ -25,6 +25,11 @@ #include <asm/proto.h> #include <asm/setup.h> +/* These are nonstandard Memory types that we do not want in the exported + * header. + */ +#define E820_UNKNOWN_12 12 /* This is the unofficial DDR3-NVDIMM */ + /* * The e820 map is the map that gets modified e.g. with command line parameters * and that is also registered with modifications in the kernel resource tree @@ -169,6 +174,9 @@ static void __init e820_print_type(u32 type) case E820_UNUSABLE: printk(KERN_CONT "unusable"); break; + case E820_UNKNOWN_12: + printk(KERN_CONT "unknown-12"); + break; default: printk(KERN_CONT "type %u", type); break; @@ -928,6 +936,7 @@ static inline const char *e820_type_to_string(int e820_type) case E820_NVS: return "ACPI Non-volatile Storage"; case E820_UNUSABLE: return "Unusable memory"; case E820_RESERVED: return "reserved"; + case E820_UNKNOWN_12: return "unknown-12"; default: return "reserved-unknown"; } }
There are multiple vendors of DDR3 NvDIMMs out in the market today. At various stages of development/production. It is estimated that there are already more the 100ds of thousands chips sold to testers and sites. All the BIOS vendors I know of, tagged these chips at e820 table as type-12 memory. Now the ACPI comity, as far as I know, did not yet define a standard type for NvDIMM. Also, as far as I know any NvDIMM standard will only be defined for DDR4. So DDR3 NvDIMM is probably stuck with this none STD type. I Wish and call the ACPI comity to Define that NvDIMM is type-12. Also for DDR4 The motivation of this patch is to be able to differentiate this NvDIMM type from a real future "unknown-reserved" type. In this patch I name type-12 "unknown-12". This is because of ACPI politics that refuse to reserve type-12 as DDR3-NvDIMM and members keep saying: "What if ACPI assigns type-12 for something else in future" [And I say: Then just don't. Please?] Signed-off-by: Boaz Harrosh <boaz@plexistor.com> --- arch/x86/kernel/e820.c | 9 +++++++++ 1 file changed, 9 insertions(+)