Proposed Quality Assurance Amendment to National Home Energy Rating Standards

Comment #1

Amendment: Quality Assurance Amendment
Page Number: 26-27
Paragraph / Figure / Table / Note: Section 904.6
Comment Type: General

Comment:Justification for Change:

Justification: The proposed changes to Chapter 9 have deleted the section on Low Volume Raters. This means that all raters, even one that does one house per year, will have to have QA performed on them. For a new business, this places an unreasonable burden on this type of rater. The QA process will cost them more than they will make on the rating. We believe the current crossed out wording concerning an inspection for once every two years should be maintained in the Standard.
 

Proposed Change:

904.6 QA of Low-Volume Raters
904.6.1 A Rating Provider may petition the QA Committee to waive the QA on-site
inspection procedure for Raters that complete less than twenty five (25) ratings per year or less than fifty (50) ratings over a two year period (Low-Volume Raters). Said petition must provide full justification for the requested exemption.
904.6.2 The Provider shall provide a set of alternative QA procedures (e.g. increased
percentage review of rating data files, digital photo documentation) that ensure that the Provider meets the intent of the on-site inspection requirement.
904.6.3 In no case shall the Provider’s alternative procedures allow less than one percent (1%) or one (1) on-site inspection, whichever is greater, based on the sum total of all ratings performed by all Low-Volume Raters who are certified by said Provider.
904.6.4 In no case shall the Provider’s alternative procedures allow a Low-Volume Rater to go more than two (2) years without a QA on-site inspection of their work.
904.6.5 If there are consistent or substantial errors discovered in the review of rating data files, an on-site inspection shall be conducted.

Comment #2

Amendment: Quality Assurance Amendment
Page Number: USA
Paragraph / Figure / Table / Note: USA
Comment Type: General

Comment:

Comment #3

Amendment: Quality Assurance Amendment
Page Number: 25-27
Paragraph / Figure / Table / Note: 904.6 & 904.4.1.3.5
Comment Type: General

Comment:

I concur with the earlier commenter that the provisions for Low-Volume raters should be retained in the Standard.

I also think a requirement to retain architectural drawings for 3 years is burdensome and excessive.  If the QA designee feels it necessary to look at the plans, I think it is reasonable for them to do so, but given that many custom homes are only available in paper format, and given that many digital plansets consist of scores of individual files that are poorly named, reviewing the architectural plans will generally be overly burdensome and expensive.

Justification for Change:

Requiring the 3-year retention of architectural plansets is overly burdensome, as is the implied expectation that QA designees will often be using them in reviews of projected ratings.  Reasoning was articulated above.

Comment #4

Amendment: Quality Assurance Amendment
Page Number: 24
Paragraph / Figure / Table / Note: 904.1
Comment Type: General

Comment:

904.1 Is still not clear to me.  I think it needs to be re-written.  The intent is not clear.  Where/how are people cheating or missing out on a learning experience?  If the intent is that I can't do QA on myself then just say that.

The last sentence seems like it should be a whole seperate number.  ie. 904.1.2  Does it not go without saying that all QA ratings must be performed/reviewed by a QA designee??   A QA designee must review a QA designee's rating??  That means to me that 2 people (both QA designees) must look at each rating for it to be QA.  CONFUSING!!

Justification for Change:

I am a QA designee.  I can offer cheaper QA for raters if I don't have to travel and do un-necessary takeoffs.  Therefore, I have in the past had raters travel to me for QA.  There are builders who have spec homes that are vacant and can be used for this very purpose.  I do not see a conflict of interest in this method.  What is the reasoning behind 904.1 and the change? 

The second sentence says "In other words, if an individual performed any part of the inspection or rating process on a home, that individual cannot be the QA Designee or Delegate performing any part of the QA process specific to that home."  This means to me that I must find a new home every time for QA.  Is that right??  The thermal bypass can't even be done by me? 

Right now I do QA for $200.  If I have to travel, do the takeoff etc it is going to be at least twice that, but probably closer to three times that. 

 

Proposed Change:

Allow QA designees to offer raters a reduced QA fee if they travel and do homes that have already been rated by the designee.  Everyone discloses to RESNET and the Provider what is going on.  In some sense, the Provider and RESNET are getting QA on the QA designee for free!  Maybe the provider has to review the QA files on that home BEFORE the QA ratings take place. 

Or, If I'm way off base, explain what the intent of 904.1 is in plain english.

Fix the last sentence so that it makes more sense.   It implies that there are QA designees who don't meet RESNET's requirements....then are they really QA designees?

Comment #5

Amendment: Quality Assurance Amendment
Page Number: 43
Paragraph / Figure / Table / Note: 913.1
Comment Type: General

Comment:

Section 913.1 states that the effective date for these changes would be August 1, 2010.  I think it is unwise to have an effective date that is prior to the completion of the public comment period.  Please choose a date in the future, so that these changes are not applied in a retro-active fashion.

Thank you,

Megan

Justification for Change:

Changes should not have an effective date prior to the completion of the public comment period.  This makes the public comments a farce.

Proposed Change:

9130.1 The effective date of these changes to the RESNET Standards shall be
OCTOBER 1, 2010.

Comment #6

Amendment: Quality Assurance Amendment
Page Number: 28
Paragraph / Figure / Table / Note: 905.2.3
Comment Type: General

Comment:

In our region, production housing projects are few and far between.  We rate custom homes almost exclusively.  Therefore, I'm not sure that we have completed more than 5 sampling or BOP projects, despite the fact that we've been raters for more than 7 years and have been providers for several years as well.  However, these project types do pop up from time to time.  

The local availability of a provider / QA that has performed 5 of a particular rating type may be slim.  This requirement may force a provider to use a QA Designee (or a rater to use a provider) located several hours from the area where ratings are performed, adding to costs unnecessarily.  

Also, inclusion of this requirement requires clarification (in RESNET Standards) regarding which parts of QA (for certain rating types) can be performed by a particular QA Designee.  For example, can a QA Designee that is qualified to QA individually rated homes perform the field QA testing for a BOP / Sampling project if a QA Designee that is qualified to QA Sampling projects completes the review of plans, models, etc...?

 

 

 

Justification for Change:

Requiring an individual to complete a minimum number ratings before becoming a QA Designee makes sense.  Requiring an individual to complete a minimum number of each type of rating to be able to QA a particular type is excessive.  

The knowledge base required for "core" QA reviews is the same regardless of the ratings type.  The additional knowledge / information required for performing QA on various rating types is not substantial enough to require a QA Designee to demonstrate proficiency VIA experience.  Knowledge of proper practices and procedures associated with various rating types is already covered on the QA test (from what I remember).  

A QA Designee that has performed QA on hundreds of custom homes, or 25 for that matter, and has already demonstrated knowledge of various practices and procedures associated with different rating types by passing the QA test, should be qualified to perform QA on any rating type.  

Forcing a rater or a provider to seek out a QA that is "qualified" (according to 905.2.3) to QA BOP's for example, my add to rating costs excessively, depending on the local / regional availability of such individuals.

Proposed Change:

  Eliminate this requirement.

Comment #7

Amendment: Quality Assurance Amendment
Page Number: 26
Paragraph / Figure / Table / Note: 904.6
Comment Type: General

Comment:

 On behalf of the Illinois Association of Energy Raters, the primary RESNET Provider for Illinois, I urge you to put the Low Volume Rater allowance back into the standard.  95% of our raters fit into this category of less than 25 ratings per year, due in large part to low demand, no utility incentives, no state programs for new construction, and low overall education of the public here.

 

 

Justification for Change:

I fear our membership will drop off substantially, meaning less bragging rights for RESNET in Illinois, and further acceleration of BPI dominance in our area.  If our members can participate in Low Volume Rater QA, their cost of doing business goes down.  If a Rater must pay for Quality Assurance yearly, even if he/she rates only 1, or zero homes, I don't see a way to justify a continued status as a rater.  This is against the best interests of RESNET, and against our struggle to prove to our state stakeholders that there is sufficient manpower to support a state-wide HERS or ENERGY STAR incentive program.

Incidentally, we have already implemented the Low Volume QA requirements.  100% of our Raters' rating files are Quality Assured, and photo documentation includes all six sides of the home, plus TBIC, blower door, duct blaster, and equipment labels.

Proposed Change:

Please keep the Low Volume Rater Quality Assurance allowance in the new standard, and help ensure the success of our state's Rating industry.

Comment #8

Amendment: Quality Assurance Amendment
Page Number: 1
Comment Type: General

Comment:

In general, I laud the efforts of RESNET to proliferate the reliability and consistency of the HERS. A more structured QA program will aid in this proliferation. However, almost all energy efficiency and retrofit programs are administered by state, county, and municipal governments, as well as local utilities. Because of this I ask that the committee remain mindful of the fact that certain micro markets exist where there are few if any programs which require or encourage the use of the HERS Index. One of these is Illinois. There is very little new construction, no existing EEP’s which make use of the index, no funding of the Energy Star builders credit, etc.
It is specifically in these markets where dedicated individuals (Raters) have been holding on by a thread, and doing their best to advance the cause of the HERS. They do this in an environment where there is little (almost no) demand for their services. The financial factors in these markets have become nearly untenable for most Raters, already. They have to maintain the same equipment, marketing costs, insurance, fees to RESNET, fees to providers, software cost, etc., that higher volume raters in other areas of the country have. In light of RESNET having already increased fees this year for marketing through its website, a substantial cost increase at this time for QA would, in my estimation be the death knell for the HERS system in our geographic market. It is hard to imagine that these dedicated individuals would be able to continue as viable (or even hopeful) business entities.
For this reason, it is recommended that RESNET not do away with the low volume rater QA provision as it currently exists. If a Rater is a low volume Rater, then by definition, he/she is struggling already. In a localized market such as ours the HERS system is still in its infancy. At this time, making tweaks to the reliability and consistency of the system should and must take a back seat to developing public awareness of what the HERS is and does. Losing many if not most of our Raters would be counterproductive to this goal. While increased structure in RESNET’s QA program is desirable and important, the timing for dropping this particular provision could not be worse for such budding markets.
Frank R. Pipal
Executive Director
Home Energy Raters of Illinois LLC
 

Comment #9

Amendment: Quality Assurance Amendment
Page Number: 25
Paragraph / Figure / Table / Note: 904.4.1.3.3
Comment Type: Technical

Comment:

904.4.1.3.3 For ratings created for new homes from architectural drawings or
plans, confirm that the Minimum Rated Features and threshold specifications for
each plan are being properly calculated and made available for verification in the
field (i.e. geometric characteristics, duct leakage and envelope leakage
thresholds). In the case of confirmed ratings, verify that actual testing data and
specification findings from the field are accurately input into the rating software
after construction is completed;

Change 'actual testing data" to verify all on site components including window sizes, floor area, orientation, etc..as defined in RESNET STandards.

Justification for Change:

Testing data is only one componet. When doing QA the designee should verify all input values not just the blower door and duct testing.

Proposed Change:

Change 'actual testing data" to verify all on site components including window sizes, floor area, orientation, etc..as defined in RESNET STandards.
 

Comment #10

Amendment: Quality Assurance Amendment
Page Number: 27
Comment Type: Technical

Comment:

904.5.2.1.2 Failure to complete a minimum of 50% of the required Quality
Assurance file and field reviews for one or more Raters certified by a Provider in
any given year;

What this is saying is as a Provider I am doing my job if I only do half what a Provider is required to do according to the Standards...ARE YOU INSANE!

FIX THIS TO 90% at least. Providers have a responsibilty to the industry.

Justification for Change:

50% is too low. Change to 90%

 What this is saying is as a Provider I am doing my job if I only do half what a Provider is required to do according to the Standards...ARE YOU INSANE!

FIX THIS TO 90% at least. Providers have a responsibilty to the industry.
 

Proposed Change:

Change to 90% and I would be ok with 95%

Comment #11

Amendment: Quality Assurance Amendment
Page Number: 27
Comment Type: Technical

Comment:

904.5.3.1 QA Designees must report all significant non-compliance issues for ratings
and their resolution to RESNET as part of the annual submission of QA results

Justification for Change:

Significant definition-need to define significant

Proposed Change:

Inspections-failure of 3 ratings and inspections in a row

Comment #12

Amendment: Quality Assurance Amendment
Page Number: 22
Comment Type: General

Comment:

Additionally, RESNET has the responsibility of accrediting Providers (Rating Providers,
BOP Providers, Sampling Providers, and Home Energy Survey Providers, and Quality
Assurance Providers)

Need to add IR Training Provider and add RESNET Training Providers

Justification for Change:

Need to include list of all providers

Proposed Change:

Include IR Training Providers, RESNET Rater Training Provider

Comment #13

Amendment: Quality Assurance Amendment
Page Number: 23
Comment Type: General

Comment:

903.21.9 “Shadowing” a Provider’s Raters or Rating Field Inspectors in the field as they
complete data collection, testing and inspections.

This would imply a QA Performed on a Rating Field Inspector can be replaced by QA of the Field Inspectors rater.

Justification for Change:

Sometimes it is necessary to QA only the Rating Field Inspector who is doing the work of a rater. The Standards are silent for QA of a Rating Field Inspector...allowing this change would now allow QA of the Field Inspector in lieu of the RATER. Is this the intent? If so, I am good with but we need to better spell out the QA process for the Rating Field Inspector which would replace the QA for the rater.

Proposed Change:

Clarification. There should be allowance of QA of Rating Field Inspector in lieu of the RATER overseeing them. Need better definition of QA Process for Field Inspector.

Comment #14

Amendment: Quality Assurance Amendment
Page Number: 24
Comment Type: Technical

Comment:

904.2 Providers are responsible for completing an annual submission of QA results to
RESNET. RESNET shall designate the date submissions are due, the content of each
submission, and the time frame for which data shall be provided, e.g. January 1st through
December 31st. Providers will have at least thirty (30) days from notification until the
submission is due.

The annual date for submission should be the same for each provider

Justification for Change:

It has been frustrating to continue to change dates for QA submissions over the year. Assigning a specific date for QA submissions for each provider is a good idea.

Proposed Change:

The annual date for submission should be the same for each provider
Each provider should have an annual QA submission date which does not change.

Comment #15

Amendment: Quality Assurance Amendment
Page Number: 26
Comment Type: Editorial

Comment:

904.4.2.5.4. Input and creation of the software rating file and reports

Not all raters perform their own electronic rating files. RESNET should allow creation or review of the software rating file and reports

Justification for Change:

Not all raters perform their own electronic rating files. RESNET should allow creation or review of the software rating file and reports
 

Proposed Change:

Creation/review

Comment #16

Amendment: Quality Assurance Amendment
Page Number: 27
Paragraph / Figure / Table / Note: 904.5.1.1
Comment Type: General

Comment:

1) It is unclear in this section what the period of time for significant non-compliance issues is.

2) Depending on the period of time, "greater than two" could be rather stringent.

Justification for Change:

To avoid confusion, the time frame must be clearly stated

Proposed Change:

For greater than two significant non-compliance issues, a 90-day time period would be more relevant.

For an annual review and time period, increase the number to greater than 5.

Comment #17

Amendment: Quality Assurance Amendment
Page Number: 27
Paragraph / Figure / Table / Note: 904.5.2.1.2
Comment Type: General

Comment:

 50% is too low. Raise to 90%

Justification for Change:

Providers are critically important in ensuring that Raters represent our industry with distinction. The levels of required reviews are not strenuous and need to be upheld to make the role of our industry secure from criticism.

Comment #18

Amendment: Quality Assurance Amendment
Page Number: 32
Paragraph / Figure / Table / Note: 906
Comment Type: General

Comment:

906 QUALITY ASSURANCE REQUIREMENTS FOR THIRD-PARTY ENERGY
EFFICIENCY PROGRAMS

This section should be removed. The relationship between a Provider, Rater, and EEP should not be considered to be under the jurisdiction of RESNET outside of the EEP requiring that relevant parties remain in good standing with RESNET. There needs to remain a seperation allowing an EEP and Provider/Rater to determine their specifc interactions.

Justification for Change:

906 QUALITY ASSURANCE REQUIREMENTS FOR THIRD-PARTY ENERGY
EFFICIENCY PROGRAMS

This section should be removed. The relationship between a Provider, Rater, and EEP should not be considered to be under the jurisdiction of RESNET outside of the EEP requiring that relevant parties remain in good standing with RESNET. There needs to remain a seperation allowing an EEP and Provider/Rater to determine their specifc interactions.
 

Comment #19

Amendment: Quality Assurance Amendment
Page Number: 26
Paragraph / Figure / Table / Note: 904.6
Comment Type: General

Comment:

The low volume clause should be left in the standards. The is new to builders and homeowners in our area. We have just joined 2 builders with Energy Star in 1.5 years and still no ratings.

Justification for Change:

The fee our Provider charges per home in QA would increase our price right out of the already tough market and possible out of business.

Proposed Change:

Leave as is

Comment #20

Amendment: Quality Assurance Amendment
Page Number: 27
Paragraph / Figure / Table / Note: Secttion 904.5.2, 904.5.3
Comment Type: General

Comment:

Sections 904.5.2.1.1 and 904.5.2.1.2 set a very low bar with respect to identifying significant non-compliance by Providers, allowing non-compliance with up to 35% of the individual requirements and allowing Providers to miss the target file and field reviews for one or more raters by up to 50%.  Under section 904.5.3, these rates of non-compliance would not even require that a QAD report them to RESNET in an annual submission of QA results because they would not be considered significant. 

Justification for Change:

The proposed rates would not even be considered as a "passing grade".

Proposed Change:

Increase the compliance threshold for reporting by the QAD under 904.5.3 to at least 80% for sections 904.5.2.1.1 and 904.5.2.1.2.

Comment #21

Amendment: Quality Assurance Amendment
Page Number: 47
Paragraph / Figure / Table / Note: Definition of Conditioned Floor Area
Comment Type: Technical

Comment:

It is unclear whether the definition of Conditioned Floor Area (CFA) includes both Drectly Conditioned Space and Indirectly Conditioned Space.  This definition will be critical as it is referenced in the Size Adjustment Factor in the ENERGY STAR HERS Target Index under version 3.

Justification for Change:

Clarification

Proposed Change:

Clarify whether Conditioned Floor Area (CFA) includes both Drectly Conditioned Space and Indirectly Conditioned Space.

Comment #22

Amendment: Quality Assurance Amendment
Page Number: 59
Paragraph / Figure / Table / Note: Definitions
Comment Type: General

Comment:

The definition for "QA Record", as referenced in Section 906.3 is undefined.

Justification for Change:

Clarification

Proposed Change:

Include a definition for "QA Record" or provide a more specific explanation of its meaning in Section 906.3.

Comment #23

Amendment: Quality Assurance Amendment
Page Number: 63
Paragraph / Figure / Table / Note: Definition of Thermal Expansion Valve
Comment Type: Technical

Comment:

It appears that the definition of Thermal Expansion Valve may be overly broad in that it may unintentionally allow a fixed orifice device to be considered a Thermal Expansion Valve.

Justification for Change:

Clarification

Proposed Change:

Refine the definition of a Thermal Expansion Valve to describe it as a device that varies the amount of refrigerant flow into the evaporator coil based on temperature and pressure...

or

clearly state in the definition that a fixed orifice is not considered a thermal expansion valve.

Comment #24

Amendment: Quality Assurance Amendment
Page Number: 25
Paragraph / Figure / Table / Note: 904.4.4
Comment Type: General

Comment:

Independent field verification of all minimum rated features could be problematic for things that cannot be inspected without some level of destruction- i.e cavity insulation

Comment #25

Amendment: Quality Assurance Amendment
Page Number: 32
Paragraph / Figure / Table / Note: 906.5
Comment Type: Editorial

Comment:

Please clarify definition of "negative results" that would trigger Provider submitting these results to EEP.

Comment #26

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: Section 904.4.1.3.1
Comment Type: Technical

Comment:

The "non biased selection process" called for is not defined.

 

Justification for Change:

Standards should not use terminology that is not well defined.

Proposed Change:

Modify Section 904.4.1.3.1 as follows:

904.4.1.3.1 Rating data files shall be selected from the entire pool of files available at the time of the review for each Rater using either a process that is designed specifically to identify known common error mechanisms or using a random selection process.

Comment #27

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: Section 904.4.1.3.2
Comment Type: Technical

Comment:

Section 904.4.1.3.2 is overly onerous. This section requires that a set of plans (and specifications) be provided to the QA Designee for each and every home that is reviewed and that the building input files be checked against these plans and specifications. Effectively this means that Raters must submit plans and specifications for each home that is rated since they cannot know a priori which of their homes will be selected by the QA Designee for review. The ability to accurately take information from plans and specifications and accurately enter it into software is part of the liability of the certified Rater. A full plans review for each data file review is overly onerous and will cause the cost of ratings to increase substantially. There is more than adequate data available to the QA Designee within the building data file to determine whether or not a rater understands what he or she is doing and is doing it largely correctly. This proposed level of detailed project review should be reserved for the on-site verification process and moved to Section 904.4.2 of this standard.

Justification for Change:

This provision should be modified to specify requirements for maintance of data file review materials for each file undergoing review.  The requirements for maintanence of architectural drawings or plans and any data collection, a.k.a. take-off forms should be removed to a new section on QA requirements for Raters.

Proposed Change:

Modify Section 904.4.1.3.2 as follows:

904.4.1.3.2 For projected and confirmed ratings, maintain detailed records of each rating data file review to include all deficiencies found and all actions taken to remediate these deficiencies for a minimum of 3 years.

Comment #28

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: 904.4.1.3.3
Comment Type: Technical

Comment:

Section 904.4.1.3.3 is overly onerous. This section is similar to section 904.4.1.3.2 in that it is over burdensome with respect to requirements that drawings, plans and specifications, along with any test data sheets from the field be submitted for each rated home since Raters cannot know a priori which of their homes will be selected by the QA Designee for review. Furthermore, this section makes requirements for threshold specification and field verification checklists that may only be appropriate for sampled homes without specifying that the requirements apply only to sampled homes. This is misleading and will lead to significant confusion for rated homes that are not part of a sampling regime. Again, this proposed level of detailed project review should be reserved for the on-site verification process and moved to Section 904.4.2 of this standard.

Justification for Change:

This section should be limited to reasonable requirements for threshold specifications for sampling purposes.

Proposed Change:

Modify Section 904.4.1.3.2 as follows:

904.4.1.3.3 For ratings created as threshold specifications for sampling purposes, ensure that Minimum Rated Feature characteristics are made available for verification in the field (i.e. geometric characteristics, duct leakage and envelope leakage thresholds):

Comment #29

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: Section 904.4.1.3.4
Comment Type: Technical

Comment:

Strick Section 904.4.1.3.4 in it entirity.

Justification for Change:

Section 904.4.1.3.4 assumes that Raters are required to produce field data collection forms and this may not be the case for all business models. It also requires confirmation of inputs for any improvement analysis. However, there is no reasonable mechanism for determining such input accuracy short of intervening between the Rater and his/her client.

Comment #30

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: 904.4.1.3.5
Comment Type: Technical

Comment:

The requirements of 904.4.1.3.5 are not appropriate for data file reviews and should be moved to a new section on certified Rater QA requirements and to section 904.4.2 on for those homes undergoing on-site verification.

Justification for Change:

Section 904.4.1.3.5 should be removed to a new section on certified Rater QA requirements. While it may be reasonable to require that certified Raters maintain architectural drawings, plans, specifications and field files for some minimum period of time on homes they rate, it is ludicrous, overbearing, and costly to require that the Rating Provider or their QA Designee keep such files on each and every home. Such document maintenance requirements on the part of Providers (or their QA Designees) should be limited to only those homes that have been subjected to on-site verification. The requirements in this section should be removed to Section 904.4.2 for homes that undergo on-site verification and to a new section on Rater QA requirements for all other cases.

Proposed Change:

Remove the requirements of this section to section 904.4.2 for homes that undergo on-site verification and to a new section on Rater QA requirements for all other homes. These records sould be maintained for a minimum of three years.

Comment #31

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: 904.4.1.3.6
Comment Type: Technical

Comment:

Section 904.4.1.3.6 is pertinent only to on-site verification procedures and should be removed to section 904.4.2.

Justification for Change:

This level of detailed HERS Index checking is only appropriate for on-site verification quality assurance.

Proposed Change:

Remove this requirement to the appropriate part of Section 904.4.2

Comment #32

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: 904.4.2.5
Comment Type: Technical

Comment:

Strike Section 904.2.5 in it entirity

Justification for Change:

A “centralized proctored rating” is defined predominantly as a training exercise and, as such, it does not provide for quality assurance is the traditional sense of the term. Quality assurance using on-site verification should always entail random, independent assessment of the Rater without his/her prior knowledge that the on-site verification event will take place.

Proposed Change:

Delete section 904.2.5 in it entirity

Comment #33

Amendment: Quality Assurance Amendment
Page Number: Section 904
Paragraph / Figure / Table / Note: 904.4.6
Comment Type: Technical

Comment:

The removal of Section 904.4.6 on low volume raters in unacceptable. Some mechanism for dealing with low volume Raters must be maintained within the standards. This section must not be completely stricken from the QA standards.

Justification for Change:

Low volume raters constitute a significant QA expense to a number of Rating Providers. Wiothout some provision to allow alternative means of providing QA for low volume raters, the cost of providing ratings must significantly increase to cover the additional burden.  Additionally, without this mechanism, small start-up rating companies will not be able to compete fairly with large existing companies. The removal of QA provisions for low volume rates could be interpreted as a restraint of trade or a violation of anti trust policies of RESNET.

Proposed Change:

Reinstate QA provisions for low volume raters.

Comment #34

Amendment: Quality Assurance Amendment
Page Number: 3
Paragraph / Figure / Table / Note: 104.2
Comment Type: Editorial

Comment:

104.2 …portions of an EEP’s certification process…

 

Justification for Change:

It’s not clear here that certification means “of the house” and not of the individual; also some EEPs may not involve certification of homes at all. Suggest deleting the word “certification”.
 

Comment #35

Amendment: Quality Assurance Amendment
Page Number: 15
Paragraph / Figure / Table / Note: 404
Comment Type: General

Comment:

404 SUNSET PROVISION
Suggest changing title of this to “TERMINATION” and replace the word “sunset” in the text with “terminate”.
 

Justification for Change:

404 SUNSET PROVISION
Suggest changing title of this to “TERMINATION” and replace the word “sunset” in the text with “terminate”.
 

Proposed Change:

404 SUNSET PROVISION
Suggest changing title of this to “TERMINATION” and replace the word “sunset” in the text with “terminate”.
 

Comment #36

Amendment: Quality Assurance Amendment
Page Number: 22
Paragraph / Figure / Table / Note: 901.1
Comment Type: Editorial

Comment:

901.1 Delete the first two sentences, and “Additionally,” at the beginning of the third. This is purely informative and does not enhance the understanding of the document; it does not belong in a normative standard.

Justification for Change:

901.1 Delete the first two sentences, and “Additionally,” at the beginning of the third. This is purely informative and does not enhance the understanding of the document; it does not belong in a normative standard.

Proposed Change:

901.1 Delete the first two sentences, and “Additionally,” at the beginning of the third. This is purely informative and does not enhance the understanding of the document; it does not belong in a normative standard.

Comment #37

Amendment: Quality Assurance Amendment
Page Number: 23
Paragraph / Figure / Table / Note: 903.2.9
Comment Type: Editorial

Comment:Proposed Change:

Replace “ “Shadowing” ” with “Accompanying”

 

Comment #38

Amendment: Quality Assurance Amendment
Page Number: 23
Paragraph / Figure / Table / Note: 903.3
Comment Type: Editorial

Comment:Proposed Change:

903.3 An accredited Rating Provider may have the right to challenge…
Change “may have” to “has”. This right is being granted in the standard, and is not at the discretion of RESNET.
 

Comment #39

Amendment: Quality Assurance Amendment
Page Number: 23
Paragraph / Figure / Table / Note: 903.4
Comment Type: Editorial

Comment:Proposed Change:

903.2 4 Significant inconsistencies or errors in the electronic files reviewed…
Change “electronic files” to “records”. Above is a list of various types of records that may be reviewed, RESNET should have the right to further investigation if there are significant inconsistencies or errors in any of them.

 

Comment #40

Amendment: Quality Assurance Amendment
Page Number: 24
Paragraph / Figure / Table / Note: 904.1
Comment Type: Editorial

Comment:

904.1 … shall be reviewed for quality assurance by an individual who meets the QA Designee…
Change “an individual” to “a separate individual”.

Also, I would suggest adding a provision (after the present language and prior to the new language, before “Any rating performed…”) that “The QA designee for a provider shall not be an individual who is responsible for production or whose compensation depends on the volume of ratings conducted by those raters being reviewed, such as a field supervisor or operations manager”. Simply stating that a QA designee may not inspect her own work is not an adequate protection against conflict.

Justification for Change:

904.1 … shall be reviewed for quality assurance by an individual who meets the QA Designee…
Change “an individual” to “a separate individual”.

Also, I would suggest adding a provision (after the present language and prior to the new language, before “Any rating performed…”) that “The QA designee for a provider shall not be an individual who is responsible for production or whose compensation depends on the volume of ratings conducted by those raters being reviewed, such as a field supervisor or operations manager”. Simply stating that a QA designee may not inspect her own work is not an adequate protection against conflict.

Proposed Change:

904.1 … shall be reviewed for quality assurance by an individual who meets the QA Designee…
Change “an individual” to “a separate individual”.

Also, I would suggest adding a provision (after the present language and prior to the new language, before “Any rating performed…”) that “The QA designee for a provider shall not be an individual who is responsible for production or whose compensation depends on the volume of ratings conducted by those raters being reviewed, such as a field supervisor or operations manager”. Simply stating that a QA designee may not inspect her own work is not an adequate protection against conflict.

Comment #41

Amendment: Quality Assurance Amendment
Page Number: 24
Paragraph / Figure / Table / Note: 904.3.2
Comment Type: Editorial

Comment:Proposed Change:

904.3.2 For the initial, first-time
Delete “, first-time” (redundant)
 

Comment #42

Amendment: Quality Assurance Amendment
Page Number: 25
Paragraph / Figure / Table / Note: 904.1.3.2
Comment Type: Editorial

Comment:Proposed Change:

904.4.1.3.2 … a.k.a. take-off forms …
Replace “a.k.a. take-off forms” with “(e.g., “take-off forms”)” (editorial)
 

Comment #43

Amendment: Quality Assurance Amendment
Page Number: 25
Paragraph / Figure / Table / Note: 904.1.3.3
Comment Type: Editorial

Comment:Justification for Change:

904.4.1.3.3
After “threshold specifications” add “, including worst-case analysis,”. Worst case analysis is a key part of determining threshold specifications, especially for sampling, and must be verified.
 

Proposed Change:

904.4.1.3.3
After “threshold specifications” add “, including worst-case analysis,”. Worst case analysis is a key part of determining threshold specifications, especially for sampling, and must be verified.
 

Comment #44

Amendment: Quality Assurance Amendment
Page Number: 25
Paragraph / Figure / Table / Note: 904.1.3.6
Comment Type: Editorial

Comment:Proposed Change:

904.4.1.3.6
Replace “what the QA Designee finds to be the accurate or true HERS Index.” with “the HERS index result as determined by the QA Designee.”
 

Comment #45

Amendment: Quality Assurance Amendment
Page Number: 26
Paragraph / Figure / Table / Note: 904.2.5
Comment Type: Editorial

Comment:

904.4.2.5 If a QA Designee is required to complete an onsite QA inspection on at
least two (2) homes for a given Rater, the QA Designee may use one centralized –
proctored rating QA event, and only one, for review of the Rater in lieu of an
independent confirmation of the rating for the home as required for the balance of
homes evaluated for the onsite inspection process.
It is entirely unclear what this means. If it means that one “proctored” QA rating may be used in place of all the required field inspections for that rater, we disagree strongly. The only thing this proctoring accomplishes is to show that the rater is capable and competent to work independently; it does nothing to show that they will honestly and accurately represent what is happening across a number of ratings in a production setting where conflicts may exist. The purpose of independent QA is to provide checks for both of these circumstances, and the latter is particularly difficult to determine without an in-person review. If this section means that a QA designee may substitute a “proctored” QA rating for one of the in-field ratings, then we certainly support that.

Comment #46

Amendment: Quality Assurance Amendment
Page Number: 27
Paragraph / Figure / Table / Note: 904.5.2.1.1
Comment Type: Editorial

Comment:Justification for Change:

Insert at the end of this sentence: “The requirement to conduct file and field reviews in accordance with 904.4.1 and 904.4.2 may not be included in the 35% of failures/omissions.” File and field reviews are central to the whole concept of RESNET QA, and cannot be avoided within the 35% of “allowable” omissions.

Proposed Change:

Insert at the end of this sentence: “The requirement to conduct file and field reviews in accordance with 904.4.1 and 904.4.2 may not be included in the 35% of failures/omissions.” File and field reviews are central to the whole concept of RESNET QA, and cannot be avoided within the 35% of “allowable” omissions.

Comment #47

Amendment: Quality Assurance Amendment
Page Number: 27
Paragraph / Figure / Table / Note: 904.2.1.2
Comment Type: Editorial

Comment:

904.5.2.1.2 Failure to complete a minimum of 50% of the required Quality
Assurance file and field reviews for one or more Raters certified by a Provider in
any given year;
 

Justification for Change:

Without requiring a high level of compliance overall, this language as proposed would effectively set the standard at half the required rating and file reviews without consequence to the QA Designee or Provider.

Proposed Change:

Change “50%” to “90%”; or, alternately change “one or more Raters” to “two or more Raters”; add at the end of the sentence: “. The file and field review rate must average 90% or more overall for the raters certified by that Provider in a given year;”

Comment #48

Amendment: Quality Assurance Amendment
Page Number: 27
Paragraph / Figure / Table / Note: 904.5.2.1.5
Comment Type: Editorial

Comment:

904.5.2.1.5 …that may be deemed by the Quality Assurance Designee to have or could have a negative impact…

Justification for Change:

It’s not clear that this clause is even necessary, and could be deleted, because the code of ethics (already covered by 904.5.2.1.3 and 904.5.2.1.4) includes a prohibition of actions injurious to RESNET or the rating industry, in rather more carefully worded language.

Proposed Change:

changing this phrase to “... that are deemed by the Quality Assurance Designee to have a negative impact or potential negative impact…” ;or delete this section entirely.

Comment #49

Amendment: Quality Assurance Amendment
Page Number: 31
Paragraph / Figure / Table / Note: 905.9.2
Comment Type: Editorial

Comment:

9045.7.89.2 “… when a certified Rater changes Providers, the new Rater’s new …”

Justification for Change:

It may be best practice to conduct these additional ratings, but it is not best practice to simply consider conducting them.

Proposed Change:

Delete the first instance of the word “new”. The provider is new, the rater is the same. Also change “…consider conducting…” to “conduct”.

Comment #50

Amendment: Quality Assurance Amendment
Page Number: 31
Paragraph / Figure / Table / Note: 905.9.3
Comment Type: Editorial

Comment:Proposed Change:

9045.7.89.3 Replace “ …what the QA Designee finds to be the accurate or true HERS Index with “the HERS index result as determined by the QA Designee.”

Comment #51

Amendment: Quality Assurance Amendment
Page Number: Appendix B
Paragraph / Figure / Table / Note: all
Comment Type: Technical

Comment:

Many or most of these definitions are new or have been modified from previous versions of the standards. This section of the proposed standard has not been properly reviewed by the RESNET Technical Committee for internal consistency or accuracy.

Justification for Change:

Improper or technically inaccurate definitions are deliterious to good standards.

Proposed Change:

Require proper technical review of Appendix B prior to publication.

Comment #52

Amendment: Quality Assurance Amendment
Page Number: 44-66
Paragraph / Figure / Table / Note: Appendix B
Comment Type: General

Comment:

Appendix B, the new definitions section, contains numerous definitions that are incorrect, inconsistent, and/or inappropriate.

Justification for Change:

Many appear to have been copied from the draft CHEA standard and are not appropriate given the currently proposed Chapter 7 and Interim Guidelines. It needs substantial work to be coordinated with all the other chapters in the RESNET standard, edited, and reviewed for technical accuracy.

Proposed Change:

We suggest that this entire section be withdrawn, separated from the rest of proposed chapter 9, re-worked substantially, and re-submitted for public comment as a separate document.
 

Comment #53

Amendment: Quality Assurance Amendment
Page Number: 25
Paragraph / Figure / Table / Note: 904.5.1.1
Comment Type: Technical

Comment:

One of the proposed changes to QAQC delivery raises concerns about the intent and fairness of the QAQC process, especially as it relates to Ratings conducted on existing homes.

904.4.1.3.6 Confirm that HERS Index scores for each file/home reviewed be no more than two (2) HERS Index points (+/-) variation from what the QA Designee finds to be the accurate or true HERS Index.

Currently, the standards address ratings found outside of acceptable modeling tolerances in terms of both HERS Index points and modeled MMBTU/yr consumption in the Energy Cost and Feature Report. QA or QC personal enforcing these items gave Rating Providers and their Quality Assurance Designee and delegates an enforcement tool to observe errors associated with both the annual estimated consumption and the HERS Index. In dispensing with the reference to allowable MMBTU/yr consumption, we feel that RESNET is allowing for greater error in modeling estimated fuel utilization, while relying upon accuracy in the HERS Index. As the HERS Index compares modeled home against geometrical code built twin, but does take into account modeled consumption, the fuel consumption reports generated out REM/Rate or other approved Rating software packages are important and used to generate UDC code compliance reports, can be used for equipment sizing, and are more often being used in the construction design phase to lower design heating, cooling and base load consumption for both new construction and retrofit projects.

Another issue that we feel could be addressed in the proposed standards is more appropriate guidance for reviewing the quality of ratings conducted on existing homes. While QAQC activities conducted on new construction ratings conducted in the Focus on Energy Programs in support of Wisconsin Energy Star Homes and EPACT tax credits have found few modeling or testing errors that have caused a reported building file to be found more than +/-2 HERS Index points from the findings of the file or field review, it may be unrealistic to expect the same margin of error in existing homes ratings. While every effort is made on the behalf of raters and the program to delivery accurate ratings, the margin of error of +/-2 HERS Index points seems to favor accuracy in new construction ratings. The HERS Index is a scale of energy efficiency between 0-500, with ratings conducted in support of Energy Star for Homes currently requiring a score of below 75 or 80, with most homes rated in our programs having a HERS index of approximately 55-65. Ratings conducted on existing homes generally have HERS Index scores from 100 to 200. It is fairly uncommon to find a home rating much above a HERS Index of 250, but this would be a possibility. For a home with HERS Index of 60, the RESNET allowable tolerance for acceptable margin of error would be around 3.3%, whereas a home rated with a HERS index of 200 would only allow for an acceptable margin of error of 1%. Furthermore, evaluating the minimum rated features of the older, inefficient home may prove substantially more difficult and time consuming, with potential for significant increase in required surface, area and volume calculations as well as additional variation in assessing insulation material present. If RESNET wishes the HERS Index to be applicable to existing housing stock and allow well intentioned, consistently accurate Raters, Rater Providers and QADs to maintain accreditation and credibility, we suggest dispensing with the +/- 2 HERS Index points and replacing with +/- 2%, or whatever percentage the technical standards committee should find acceptable for an acceptable margin of error for the Home Energy Rating System.
 

Justification for Change:

See above comment.

Proposed Change:

904.4.1.3.6 Confirm that HERS Index scores for each file/home reviewed be no more than two (+/-) 3% variation from  the HERS Index score that the QA Designee finds to be the accurate or true HERS Index.

Confirm that the modeled consumption in the Energy Cost and Feature Report for each file/home reviewed be no more than two (+/-) 3% variation from the modeled consumption in the Energy Cost and Feature Report that the QA Designee finds to be the accurate or true modeled consumption.

Comment #54

Amendment: Quality Assurance Amendment
Page Number: 3
Paragraph / Figure / Table / Note: 102.1.2.2
Comment Type: General

Comment:

It appears from the change in language that only confirmed ratings will be allowed during a Rater's probationary phase. We agree that probationary ratings should evaluate a new rater’s ability to gather data and create a rating on existing homes, but we still see value to having a probationary rating performed from plans. These are two unique skills that we feel are necessary for all raters.

Justification for Change:

From-plans ratings are a critical role for Raters to perform. Because of this we think it is important that Rater's complete a from-plans rating during the probationary phase of certification. In addition, from-plans ratings allow providers to have a single streamlined review process that allows quicker turnaround times. In the end it benefits both parties by practicing a critical role for the probationary Rater while reducing review times of the provider.

Proposed Change:

Make it mandatory that two of the probationary ratings are confirmed with an optional from-plans rating. RESNET can provide the plans for providers to utilize during the probationary phase. This will ensure that the plan is of sufficient quality.

Comment #55

Amendment: Quality Assurance Amendment
Page Number: 25
Paragraph / Figure / Table / Note: 904.4.1.3.2
Comment Type: General

Comment:

We are worried about potential problems collecting plans. Builders/architects need to be notified by the rater that their plans may be submitted to the HERS provider for the purposes of QA. 

Comment #56

Amendment: Quality Assurance Amendment
Page Number: 26
Paragraph / Figure / Table / Note: 904.4.2.5
Comment Type: General

Comment:

This section seems to benefit high-volume Raters with an option to do a proctored session. While this may be appealing to the high-volume Rater, it in no way helps the low-volume Rater that proctoring was intended to help. In light of striking the low-volume exemption from the standards we propose that this centralized-proctored rating QA event be allowed for low-volume Raters.

Justification for Change:

The program we're proposing is more beneficial to the low-volume Rater than the high-volume Rater. Low-volume Raters tend to need the feedback and discussion provided at the end of proctored sessions more than those that are in the field performing ratings on a daily basis.

Proposed Change:

By providing low-volume Raters with a proctored session we are improving their capability to provide high quality ratings. If a low-volume Rater receives a proctored rating in a given year then they will be required to have a blind quality assurance visit in the following year. This strengthens the current low-volume exemption program by ensuring that the provider is having contact with those low-volume Raters on a yearly basis.

Comment #57

Amendment: Quality Assurance Amendment
Page Number: 28
Paragraph / Figure / Table / Note: 905.2.3
Comment Type: Editorial

Comment:

The section seem to be mispunctuated or impcomplete. Is EEP a rating type?

Return to Proposed Quality Assurance Amendment to National Home Energy Rating Standards