Minutes of Study Group Meeting, 2017-12-11

Meeting called to order: 11:07 AM EST

The slide references relate to the pack used during this meeting, located here: http://files.sjtag.org/StudyGroup/SG_Meeting_17.pdf

Papers supplied by Duane Lowenstein have been circulated by email: "Keysight papers shared with the SJTAG group".
Papers referenced by Louis Ungar have been circulated by email: "Louis' papers for the SJTAG group".

1. Roll Call

Ian McIntosh (Leonardo MW Ltd.)
Heiko Ehrenberg (Goepel Electronics)
Eric Cormack (DFT Solutions Ltd.)
Terry Duepner (National Instruments)
Bill Eklow (Retired)
Brian Erickson (JTAG Technologies)
Peter Horwood (Firecron Ltd.)
Bill Huynh (Marvell Inc.)
Joel Irby (ARM) (joined 11:12)
Teresa McLaurin (ARM)
Naveen Srivastava (Nvidia)
Jon Stewart (Dell)
Brad Van Treuren (Nokia)
Carl Walker (Cisco Systems)
Ed Gong (Intel Corp.)
Russell Shannon (NAVAIR Lakehurst)
Louis Ungar (ATE Solutions)
Duane Lowenstein (Keysight)

By email (non-attendees):

Mukund Modi (NAVAIR Lakehurst)

2. IEEE Patent Slides

  • {Slides 5-9}

3. Review and Approve Previous Minutes

  • {Slide 10}
  • December 4
    • Draft circulated 12/04/17
    • No corrections noted.
    • Brian moved to approve, seconded by Brad, no objections or abstentions. Approved.

4. Review Open Action Items

  • {Slide 11}
  • [10.2] Brad will draft a definition for "boundary".
    • ONGOING.
  • [14.1] ALL: Develop Purpose description.
    • ONGOING.
  • [16.1] Jon: Enquire if there is data on NFF rates in higher volume product areas (e.g. laptops).
    • Looked at server boards: General sense is that the highest rate of NFFs arise from interconnects (plug or pin and socket, e.g. CPUs and DIMMs). Looking to put in place some form of JTAG solution with loopbacks that doesn't use the CPU to test the board.
    • Is there any threshold of NFFS from the field that triggers developers to get involved? Focus wasn't on field returns. Some sense that NFFs are a bigger problem with field returns.
    • There are two types of NFF: One where the cause was never tested in the first place, the other where it arises from measurement uncertainty and margin stack-up. Could also include false alarms and design marginality here.
    • {Duane supplied a paper, "Tighter Design Margins", to Ian via email, for circulation to the group}
    • Reported to be cases where warranties are not honoured for NFFs.
    • Differences between instruments can significantly affect results;, as well as heating/cooling, adding fixtures, switch matrices, cabling, etc.
    • Is there a test accuracy ratio where you can say a measurement is going to be good? SystemVue would allow you to take several layers of abstraction from the DUT and put it all together to assess.
    • NAVAIR also looking at interconnects (at the backplane) as a significant fault source.
    • {Louis supplied a paper post-meeting on costs and causes of NFFs, IPC, 2015}
  • [16.2] Jon/Louis: Approach Duane Lowenstein to enquire whether Keysight have anything on system modelling.
    • (Discussion flowed into 5b, below).

5. Discussion Topics

a)  Refocus/quick refresh on what the study group is expected to deliver to TTSC.

  • {Slide 12}
  • (This section was addressed after 5b due to the way discussions developed from the review of actions).
  • Presented as a reminder of what the group should be doing.
  • Study Group guidelines: http://files.sjtag.org/StudyGroup/studygrp.pdf. 
  • Group is expected to consider:
    • Potential market acceptance of the standards project, including technical feasibility
    • Relationship to related standards, if known, including its distinct identity from other projects
    • Viable volunteer leadership and participation
    • Realistic scope and objectives
    • Opportunity to establish new areas of expertise within the Sponsor

b) Scope, Purpose and Need:
Continue developing concepts of what the standard(s) needs to address.

  • {Slide 13}
  • There are no metrics for system level faults. It may be that different companies will each look for different things from them anyway. In many cases the data isn't available to help work out the cost of test, or is at least hard to get to. Measurement uncertainty and margin stack-up are really the things that are more manageable and will come into play.
  • You can attempt to minimise NFFs but you will never completely eliminate them.
  • Extensive data might be collected but often that's all that is done; there is no way to interrogate that data to find out what might be a common occurrence. There's a reliance on the people recording fault data to do it consistently.
  • We have fault models for chips, why can't we have fault models for systems?
  • You may need a multitude of fault models to cover all the different configurations that might be possible. The designer may not have full control over the configurations the end-user employs.
  • PCOLA-SOQ was proposed for components on PCBs: An "extravagant" measure at it uses 8 qualifiers. Agilent's Fault Detective was simpler and looked at the coverage of function in terms of the coverage itself and a measure of ambiguity in the diagnostic. This was probably workable at the board level but might become a bit unwieldy at system level.
  • {Duane supplied a paper, "Built-in Test -Coverage and Diagnostics", to Ian via email, for circulation to the group}
  • Louis had proposed extending PCOLA-SOQ, first as "FAM" - Function, At-speed and Measurement, then "STRID" {IEEE paper, 2010, circulated post-meeting}.
  • May be looking for something that needs high volumes in order to obtain the metrics. Not necessarily - the paper looks at testability metrics for COTS - what you might want to know when selecting a COTS item: Is there a way for me to test and diagnose this in my system? Testability metric, not a test metric.

6. Today's Key Takeaways

  • {Slide 14}
  • We are not addressing what constitutes a "good" versus a "bad" system.
    • This may be needed for "system test" but is not necessarily within the remit of "System Test Access Management". We need the access first, then we can work on  what the test does. Concern that we are "expanding our horizons".
    • Louis will look to see if any relevant standards exist {ACTION}.
    • May need to do some partitioning - there will be some things that we need to do for the standard, others that will be supportive or optional.

7. Glossary Terms from This Meeting

  • None.

8. Topic for next meeting

  • TBD (will be advised in the Calling Notice on Wednesday).

9. Schedule next meeting

  • December 18.
    • Brad and Terry will be out. 
    • Dec. 18 is the last meeting date this year, resume on January 8.

10. Reminders

  • None.

11. Any Other Business

  • None.

12. List New Action Items

  • [17.1] Louis: Search for any standards that address what constitutes a fault (or pass) at system level.

13. Adjourn

  • Terry moved to adjourn, seconded by Eric.
  • Meeting adjourned at 12:01 PM EST

Respectfully submitted,
Ian McIntosh