nfl health & safety helmet assignment

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

NFL Health & Safety - Helmet Assignment: Segment and label helmets in video footage

regisss/nfl_helmet_assignment

Folders and files, repository files navigation, project setup.

One can setup the project with the following command (preferably in a virtual environment):

Helmet detection notebook

The notebook yolov5_helmet_detection.ipynb displays an application of YOLOv5 to helmet detection. Running the whole notebook will generate a video clip with inference results obtained on one of the test videos. This video clip can be viewed in the last cell of the notebook.

Be careful to specify well where you store Kaggle's data for this challenge (in the second code cell of the notebook).

The goal of this competition is to assign specific players to each helmet.

In order to do so, the targeted pipeline of this project is the following:

• Helmet detection with an object detection algorithm such as YOLOv5. There may be issues with sideline players' helmets.
• Helmet tracking so that several different instances do not switch with each other (SORT algorithm?). As there may be many players in a very small area, this can be challenging.
• Use the provided tracking data to identify players.
• Jupyter Notebook 99.9%

• I want to launch a challenge Connect to talented problem solvers
• I want to solve a challenge Join our community to win prizes

NFL Health & Safety - Helmet Assignment

The National Football League (NFL) and Amazon Web Services (AWS) are teaming up to develop the best sports injury surveillance and mitigation program. In previous competitions, Kaggle has helped detect helmet impacts. As a next step, the NFL wants to assign specific players to each helmet, which would help accurately identify each player's “exposures” throughout a football play.

Currently, the NFL manually annotates a subset of plays each year to determine a sample of exposures for each player. To expand this program, the current player assignment requires a field map to determine player locations. The NFL is interested in matching this model's accuracy without the need for the mapping step. The league is calling on Kagglers to invent a better way to identify individual players.

The National Football League is America's most popular sports league. Founded in 1920, the NFL developed the model for the successful modern sports league and is committed to advancing progress in the diagnosis, prevention, and treatment of sports-related injuries. Health and safety efforts include support for independent medical research and engineering advancements as well as a commitment to work to better protect players and make the game safer, including enhancements to medical protocols and improvements to how our game is taught and played. For more information about the NFL's health and safety efforts, please visit www.NFL.com/PlayerHealthandSafety .

In this competition, you’ll identify and assign football players’ helmets from video footage. In particular, you'll create algorithms capable of assigning detected helmet impacts to correct players via tracking information. Successful submissions should aim for 90% accuracy.

If successful, you'll support the NFL in its efforts to efficiently improve player safety. If the league no longer has to manually label each exposure, it would dramatically increase the speed and scale at which they could answer complex research questions related to helmet impact. Automatic player detection would also allow the NFL to back-calculate historic exposure trends, allowing for deeper insights into how to mitigate them in the future.

One account per participant

You cannot sign up to Kaggle from multiple accounts, and therefore, you cannot submit from multiple accounts.

No private sharing outside teams

Privately sharing code or data outside of teams is not permitted. It's okay to share code if made available to all participants on the forums.

Team Mergers

Data Science Team mergers are allowed and can be requested by the team leader. In order to merge, the combined team must have a total submission count less than or equal to the maximum allowed as of the requested merge date. The maximum aggregate number of submissions allowed by the combined team as of the merger date is the number of allowable submissions per day multiplied by the number of calendar days starting from the Start Date until the date of such merger, and the aggregate number of Submissions made by all of the combined team members does not exceed such amount.

Team Limits

The maximum team size is 5.

Submission Limits

You may submit a maximum of 5 Submissions per day.

You may select up to 2 final submissions for judging.

Competition Timeline

Competition Timeline dates (including Entry Deadline, Final Submission Deadline, Start Date, and Team Merger Deadline, as applicable) are reflected on the competition’s Overview > Timeline page.

COMPETITION-SPECIFIC TERMS

COMPETITION TITLE : NFL Helmet Assignment

COMPETITION SPONSOR : The National Football League

COMPETITION SPONSOR ADDRESS : 345 Park Avenue, New York, NY 10154

COMPETITION WEBSITE : https://www.kaggle.com/c/nfl-health-and-safety-helmet-assignment

TOTAL PRIZES AVAILABLE: $100,000

First Prize: $50,000

Second Prize: $25,000

Third Prize: $13,000

Fourth Prize: $7,000

Fifth Prize: $5,000

WINNER LICENSE TYPE : Non-Exclusive License

DATA ACCESS AND USE : Competition-Use Only Competitions are open to residents of the United States and worldwide, except that if you are a resident of Crimea, Cuba, Iran, Syria, North Korea, Sudan, or are subject to U.S. export controls or sanctions, you may not enter the Competition. Other local rules and regulations may apply to you, so please check your local laws to ensure that you are eligible to participate in skills-based competitions. The Competition Sponsor reserves the right to award alternative Prizes where needed to comply with local laws.

ENTRY IN THIS COMPETITION CONSTITUTES YOUR ACCEPTANCE OF THESE OFFICIAL COMPETITION RULES.

The Competition named above is a skills-based competition to promote and further the field of data science. You must register via the Competition Website to enter. Your competition submissions (“Submissions”) must conform to the requirements stated on the Competition Website. Your Submissions will be scored based on the evaluation metric described on the Competition Website. Subject to compliance with these Official Rules, Prizes, if any, will be awarded to participants with the best scores, based on the merits of the data science models submitted. See below for the complete Official Rules.

NO PURCHASE NECESSARY TO ENTER OR WIN.

A PURCHASE DOES NOT IMPROVE YOUR CHANCES OF WINNING.

THIS COMPETITION IS GOVERNED EXCLUSIVELY BY THE LAWS OF THE UNITED STATES.

Please read these rules before entering the Competition. Participation in this Competition constitutes each Participant’s full and unconditional agreement to and acceptance of these Official Rules and represents that the Participant satisfies all of the eligibility requirements set forth below.

1. EXECUTIVE SUMMARY: NFL Helmet Assignment (the “Data Science Theme”) is an open, skills-based competitive event; a description of which is below:

Data Science Theme – NFL Computer Vision Competition

2. ELIGIBILITY: To be eligible to enter the Competition, each Participating Person entering the Competition must be:

(i) A legal resident of the country in which he or she resides; (ii). At least 18 years old or the age of majority in such individual's jurisdiction of residence; (iii). Not a resident of Crimea, Cuba, Iran, Syria, North Korea, or Sudan; and (iv). Not a person or representative of an entity under U.S. export controls or sanctions (see https://www.treasury.gov/resource-center/sanctions/Programs/Pages/Programs.aspx ).

All applicable United States federal, state, provincial and local laws and regulations apply. Although the Competition is governed exclusively by the laws and regulations of the United States, other local rules and regulations may apply to certain Participating Persons, so all Participating Persons should check their local laws to ensure that he or she is eligible to participate in skills-based competitions. The Sponsor and Named Partners reserve the right to award alternative prizes where needed to comply with local laws.

If any Participating Person is entering as a representative of a company, educational institution or other legal entity, or on behalf of his or her employer, these rules are binding on such Participating Person, individually, and the entity such person represents or are an employee of. If any Participating Person is acting within the scope of his or her employment, as an employee, contractor, or agent of another party, such Participating Person warrants that such party has full knowledge of his or her actions and has consented thereto, including his or her potential receipt of a prize. Such Participating Person further warrants that his or her actions do not violate his or her employer's or entity's policies and procedures.

Employees of the Sponsor, Named Partners, the Administrators or any person involved in the production, development, implementation or handling of the Competition, any agents acting for or on behalf of the above entities, their respective parent companies, officers, directors, subsidiaries, affiliates (including, in the case of the NFL, the NFL's member professional football clubs), licensees, sponsors, service providers, prize suppliers or any other person or entity associated with the Competition (collectively, including the Sponsor, the Named Partners and Administrators, the " Sponsor-Related Persons ") and/or the immediate family (spouse, parents, siblings and children) and household members (whether related or not) of each such employee, are eligible to enter and participate in the Competition but are not eligible to win any prizes.

The Data Science Theme is open to teams of one (1) or more, but not in excess of five (5), data scientists or researchers (each such person, a "Participant") provided, however , that each Participant may join or form only one team. Each team that enters the Data Science Theme will be referred to herein as a " Data Science Team ". Each Participant must be a single registered account holder at Kaggle.com. Any Participant must register individually for the Data Science Theme before joining a Data Science Team. Such individual must make his or her registration official by accepting the Official Rules on Kaggle through his or her Kaggle account. All applicable Participants will be disqualified if he, she or it, as applicable, makes a Data Science Submission through more than one Kaggle account or attempts to falsify an account to act as such Participant's proxy.

All Data Science Teams are permitted to request to combine two or more Data Science Teams into one Data Science Team (such proposed combined Data Science Team, the " Combined Data Science Team ") via the Data Science Site (defined below). Data Science Team mergers shall be permitted provided that, (i) such Combined Data Science Team does not exceed the maximum number of five (5) Participants, (ii) the maximum aggregate number of submissions allowed by the Combined Data Science Team as of the merger date is the number of allowable submissions per day multiplied by the number of calendar days starting from August 10, 2021 until the date of such merger, and the aggregate number of Submissions made by all of the Data Science Team members does not exceed that amount, (iii) the merger is completed by no later than 11:59 PM UTC on October 26, 2021, and (iv) the proposed Combined Data Science Team otherwise meets all of the requirements set forth in these Official Rules.

The Sponsor and Named Partners reserve the right to verify eligibility and to adjudicate on any dispute at any time. If any Participant provides any false information relating to the Competition concerning his, her or its, as applicable, identity, residency, mailing address, telephone number, email address, ownership of right, or information required for entering the Competition, such Participant may be immediately disqualified from the Competition. If a Competition winner is subsequently discovered to be ineligible, the Sponsor and Named Partners reserve the right to forfeit/reclaim any Competition prizes at their discretion, and confer the same on such other Participant as may be selected by the Sponsor and Named Partners in their sole discretion.

3. SPONSORS; ADMINISTRATORS: The Competition is sponsored by the National Football League, with offices at 345 Park Avenue, New York, New York 10154 (the " NFL " or " Sponsor "), and a third-party business partner that may be identified (the " Named Partners ").

The Data Science Theme is administered and hosted on behalf of the Sponsor and Named Partners by the NFL and Kaggle Inc. (" Kaggle ", the " Administrator " with respect to the Data Science Theme). Kaggle is an independent contractor of the Sponsor and is not a party to this or any agreement between any Participant and the Sponsor or Named Partners, including these Official Rules. Kaggle will perform certain administrative functions relating to hosting the Data Science Theme. Each Participant understands that Kaggle has no responsibility with respect to selecting the potential Competition winner(s) or awarding any Prizes. As a Kaggle.com account holder and user of the Kaggle competition platform, each Participant acknowledges and agrees that he, she or it, as applicable, has accepted and is subject to the Kaggle Terms of Service at www.kaggle.com/terms in addition to these Official Rules.

4. AGREEMENT TO OFFICIAL RULES: Each Participant should read these Official Rules carefully before entry to ensure that such Participant understands and agrees.Participation in the Competition will require each Participant to accept, prior to submission, these Official Rules, which constitutes each Participant's full and unconditional agreement to and acceptance of these Official Rules and the decisions of the Sponsor and Named Partners, which are final and binding. Winning a prize is contingent upon fulfilling all requirements set forth herein. The Sponsor and Named Partners reserve the right to take any actions necessary to verify a Participant's compliance with these Official Rules before awarding a prize, including, without limitation, engaging a third party to evaluate the Submission and/or requiring a Participant to provide evidence of permission to use certain third-party materials. Even though a Participant may be announced as a winner, if any such Participant's compliance with these Official Rules cannot be verified to the Sponsor's or Named Partners' satisfaction, the Participant will be disqualified, and, time permitting, an alternate winner will be selected.

5. ENTRY: The Data Science Theme will run from the applicable start date and end date/time, as set forth below (the end date/time, the " Competition Deadlines "). The Competition Deadlines are subject to change, and the Sponsor and Named Partners may introduce additional hurdle deadlines during the Competition. Any deadline changes or additional hurdle deadlines will be publicized on the Data Science Site, as applicable. It is each Participant's responsibility to check the Data Science Site, as applicable, regularly to stay informed of any changes.

Each Submission must adhere to the Guidelines and Restrictions described below (the " Guidelines and Restrictions "). The Sponsor and Named Partners, in their sole discretion, may disqualify any Participant from the Competition if they believe that the Submission fails to conform to the Guidelines and Restrictions.

Guidelines and Restrictions : Each Submission must:

• Not contain material that violates or infringes any rights of any other party, including but not limited to copyright, trademark, patent, privacy, publicity or any other intellectual property rights;
• Not disparage any Sponsor, Named Partner or any other person or party;
• Not contain material that is inappropriate, indecent, obscene hateful, tortious, defamatory, slanderous or libelous;
• Not contain material that promotes bigotry, racism, hatred or harm against any group or individual or promotes discrimination based on race, gender, religion, nationality, disability, sexual orientation or age;
• Comply with any applicable laws or any other regulation, guideline or community standard in entrant's country of residence and must not contain material that is unlawful, in violation of or contrary to the laws or regulations of entrant's country of residence or any jurisdiction where the Submission is created; and
• Not have been developed or created for or on behalf of a client of a Participant and must not have won any kind of award, competition or contest.

Data Science Theme entries may be submitted at https://www.kaggle.com/c/nfl-health-and-safety-helmet-assignment (the " Data Science Site ") (the entry, along with all other materials provided, the " Data Science Submission "; any Data Science Submission may herein be referred to as an " Submission "). The Data Science Theme is officially launched on August 10, 2021 and to enter the Data Science Theme, Participants must register on the Data Science Site prior to October 26, 2021 (" Entry Date "). Submissions may be submitted during the period beginning on the date that such Participant registered on the Data Science Site and ending at 11:59:59 pm UTC on November 2, 2021; provided, that, for the avoidance of doubt, such Participant cannot register for the Data Science Theme later than October 26, 2021. Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

Participants of the Data Science Theme may submit up to the maximum of five (5) Submissions per day as specified on the Data Science Site. Each Data Science Team may select up to two (2) final submissions for judging.

Participants and Data Science Teams are permitted to use automated machine learning tool(s) (" AMLT ") (i.e., Google AutoML, H2O Driverless AI, etc.) in the creation of the Submissions for purposes of the Data Science Theme so long as such Participants and Data Science Teams ensure that they have an appropriate license to the AMLT and are able to comply with these Official Rules. Subject to the terms of these Official Rules, Participants and Data Science Teams that create a Submission using an AMLT may be eligible to win a prize in the Data Science Theme; provided , that such potential Data Science Winner's Submission must still meet the requirements of these Official Rules.

See Data Science Site for submission instructions. Each Data Science Submission must follow the instructions, including the manner, format and other requirements, for developing and entering the submission.

EACH PARTICIPANT IS RESPONSIBLE FOR DETERMINING THE CORRESPONDING TIME IN HIS, HER OR ITS, AS APPLICABLE, TIME ZONE. ENTRIES ARE VOID IF THEY ARE IN WHOLE OR IN PART ILLEGIBLE, INCOMPLETE, DAMAGED, ALTERED, COUNTERFEIT, OBTAINED THROUGH FRAUD, OR LATE. THE SPONSORS RESERVE THE RIGHT TO DISQUALIFY ANY ENTRANT WHO MAKES AN ENTRY THAT DOES NOT MEET THE REQUIREMENTS SET FORTH ON THE DATA SCIENCE SITE, AS APPLICABLE.

By entering, each Participant represents and warrants the following with respect to his, her or its, as applicable, entry: (a) the Data Science Team is the sole and exclusive owner of the Submission (including all concepts, materials and/or inventions included therein); (b) the Submission does not violate any (i) of the Guidelines and Restrictions set forth herein, (ii) requirements set forth in the Data Science Site, as applicable, or (iii) rights of any third parties; (c) to the extent possible under applicable law, no other party can invoke any moral rights in relation to the Submission that have not been duly waived; (d) the Submission and the participation in the Competition by the Participant do not violate any local, state, provincial, national or foreign law; and (e) the Submission does not contain the confidential information of any third party.

Each Participant acknowledges that other Participants from other entities or teams may have used ideas and/or concepts in their Submission that may be similar in idea or concept to what is included in such Participant's Submission. Each Participant understands and agrees that he, she or it, as applicable, will not have any claim against any other Participant, the Sponsor, any Named Partner or Administrator arising out any such similarity or be entitled to any compensation because of any such similarity.

6. SELECTION OF FINALISTS & WINNERS:

Up to five (5) Data Science Teams will be selected as winners (collectively, the " Data Science Winners "). All Data Science Winners may have the opportunity to appear on the virtual event and present their winning entries either, at the Sponsor's and Named Partners' sole discretion, through a pre-recorded video or interview, or via a live-stream video that may be recorded and aired at a later date as part of the virtual event. One (1) Data Science Winner will be named grand prize winners (the " Data Science Grand Prize Winner "), one (1) Data Science Winner will be named second prize winner (the " Data Science Second Prize Winner "), one (1) Data Science Winner will be named second prize winner (the " Data Science Third Prize Winner "), one (1) Data Science Winner will be named second prize winner (the " Data Science Fourth Prize Winner ") and one (1) Data Science Winner will be named second prize winner (the " Data Science Fifth Prize Winner ") as part of the virtual event.

Each Data Science Theme Submission will be scored and ranked by the evaluation metric stated on the Data Science Site. During the Competition, the current ranking will be visible on the Data Science Site's public leaderboard. The Data Science Theme potential winner(s) are determined solely by the leaderboard ranking on the private leaderboard, subject to compliance with these Official Rules. The public leaderboard will be based on the public test set and the private leaderboard will be based on the private test set.

In the event of a tie for any Data Science Theme Submissions, the Submission that was entered first to the Data Science Theme will be the winner. In the event a potential Data Science Winner is disqualified for any reason, the Submission that received the next highest score rank will be chosen as the potential winner with respect to the Data Science Theme.

A disqualified Data Science Theme Participant may be removed from the leaderboards, at Kaggle's sole discretion. If a Participant in the Data Science Theme is removed from such leaderboard, additional winning features associated with the Kaggle platform, for example, Kaggle points or medals, may also not be rewarded.

Potential winners will be notified by email. If a potential winner does not respond to the notification attempt within twenty-four (24) hours, then such potential winner may be disqualified, and an alternate potential winner will be selected from among all eligible entries received.

The final winner's list will be publicly displayed at Kaggle.com.

As a condition to being awarded a prize in the Data Science Theme, each Data Science Winner must fulfill the following obligations:

i. Deliver to the Sponsor and Named Partners the final model's software code as used to generate the winning Submission and associated documentation. The delivered software code should follow the document guidelines set forth on https://www.kaggle.com/WinningModelDocumentationGuidelines , must be capable of generating the winning Submission, and contain a description of resources required to build and/or run the executable code successfully. If the final model's software code includes generally commercially available software that is not owned by such Participant but that can be procured by the Sponsor and Named Partners without undue expense, then instead of delivering the code for that software to the Sponsor and Named Partners, must identify that software, method for procuring it, and any parameters or other information necessary to replicate the winning Submission.

ii. Grant to the Sponsor and Named Partners the license to the winning Submission, in accordance with Section 10 of these Official Rules.

iii. Sign and return all price acceptance documents as may be required by the Sponsor, the Named Partners or Kaggle, including, without limitation, eligibility certifications, licenses, releases and other agreements required by these Official Rules. Such price acceptance documents must be returns to the Sponsor, the Named Partners or Kaggle within two (2) weeks following notification of such required documents, or such potential Data Science Winner will be deemed to have forfeited the prize and another potential Data Science Winner will be selected.

The Sponsor and Named Partners reserve the right to disqualify any Participant from the Competition if the Sponsor or Named Partners reasonably believe that the Participant has attempted to undermine the legitimate operation of the Competition by cheating, deception, or other unfair playing practices or abuses, threatens or harasses any other Participants, Sponsor, Named Partners or Administrators.

Each Participant agrees that the identity of each of the Data Science Winners, shall be held strictly confidential, and Participants shall not be permitted to provide or communicate such information (or any part thereof) to any other person or entity without the express prior written consent of the NFL or, if earlier, at such time as such information is made generally available to the public by the Sponsor, the Named Partners or Administrators.

7. GRAND PRIZES:

The Grand Prize Winner will receive $50,000 in cash. The Data Science Second Prize Winners will receive $25,000 in cash. The Data Science Third Prize Winner will receive $13,000 in cash. The Data Science Fourth Prize Winner will receive $7,000 in cash. The Data Science Fifth Prize Winner will receive $5,000 in cash.

If a Data Science Team wins any of the above cash prizes, the prize money will be allocated in even shares between the eligible Data Science Team members, unless such Data Science Team unanimously requests, in writing, for a different prize split and notifies Kaggle, the Sponsor and the Named Partners before such cash prizes are issued.

Winning teams are solely responsible for any and all applicable fees and taxes associated with prize receipt and use. All federal, state, provincial and local taxes and unspecified expenses (including social contributions and/or VAT Taxes, where applicable) are the responsibility of winning teams. AWARD OF PRIZES TO POTENTIAL WINNING TEAMS ARE SUBJECT TO THE EXPRESS REQUIREMENT THAT THEY SUBMIT TO SPONSORS AND, SOLELY WITH RESPECT TO DATA SCIENCE WINNERS, KAGGLE, ALL DOCUMENTATION REQUESTED BY SPONSORS AND KAGGLE TO PERMIT THEM TO COMPLY WITH ALL APPLICABLE FEDERAL, STATE, PROVINCIAL, LOCAL OR OTHER TAX REPORTING LAW OR REGULATIONS IN THE UNITED STATES AND IN THEIR RESPECTIVE JURISDICTION. TO THE EXTENT PERMITTED BY LAW, ALL TAXES IMPOSED ON PRIZES ARE THE SOLE RESPONSIBILITY OF THE WINNERS. Refusal by any Entrant to submit such documentation or complete any required forms or obligations will result in such winner forfeiting the prize, leaving it unclaimed.

Finalists/winners are responsible for any federal, state and local taxes and fees associated with receipt or use of a prize.

ADDITIONAL DOCUMENTS / CONSENT: Except where prohibited, each Data Science Winner and their respective Participating Persons will be required to sign and return to the Sponsor and Named Partners, by a deadline to be determined, a declaration of eligibility, liability/publicity release, U.S. tax forms (such as IRS Form W-9 if U.S. resident, IRS Form W-8BEN if foreign resident, or future equivalents), licenses, releases and other agreements required under Section 8 of these Official Rules, and additional documents that may be required by the Sponsor and Named Partners in order to proceed in the Competition. Failure to return required documents as specified will result in disqualification. The declaration of eligibility, and the acceptance of any prize, will also include consent to use each Participant's name and likeness for editorial, advertising, and publicity purposes without additional compensation, except where prohibited by law.

8. INTELLECTUAL PROPERTY :

All patent, copyright and trademark rights (collectively, " Intellectual Property Rights ") belonging to any person (e.g.,a Data Science Team, Administrator, Sponsor or a Named Partner) prior to the Competition will remain vested in that person. Subject to the license created below with respect to the Data Science Winners, any Intellectual Property Rights created or otherwise developed by a Participant during the course of the Competition will vest in such Participant. Any Intellectual Property Rights created or otherwise developed by the Sponsor or a Named Partner during the course of the Competition will vest in such Sponsor or Named Partner, as applicable. Each Participant represents and warrants that the products and services discussed in its Submission are and will be the original work of and solely owned by Data Science Team, or, if a part of those products and services are not original to or solely owned by such Data Science Team, then the Participants have all necessary rights and licenses from any third party in order to incorporate such part into the products and services discussed in the Submission and as otherwise contemplated in these Official Rules. Further, each Participant represents and warrants that neither the Submission, nor any products or services discussed in the Submission, infringe or misappropriate any Intellectual Property Rights or other rights of third parties.

Each Participant acknowledges that the Sponsor's and Named Partners' consideration of the Submission is not an admission by the Sponsor and Named Partners of the novelty, propriety, originality or value of the Submission and/or application, or the products or services discussed in the Submission. Each Participant further acknowledges that the Sponsor or Named Partners may be creating, have previously created or may in the future independently create, or already may have received or in the future may receive from another third party, products, services, projects, ideas, designs and other materials that are substantially similar, identical, or otherwise related to the products or services discussed in the Submission, which the Sponsor or Named Partners may use for any purpose without any liability or compensation to any Participant. Each Participant further acknowledges that due to the nature of this Competition, there is a possibility that similar products or services may be submitted by multiple Participants. Any similarity between products or services will in no way entitle any Participant to any consideration or compensation from the Sponsor or Named Partners, including in the event a product or service similar or identical to Participant's is selected as a winner or finalist. By entering the Competition and submitting an application, each Participant specifically acknowledges this possibility and agrees to the terms stated in these Official Rules. The Sponsor and Named Partners are further under no obligation of any kind to any Participant unless such obligations are specifically undertaken pursuant to a written agreement fully executed by one or multiple Participant(s), on the one hand, and the applicable Sponsor or Named Partner(s), on the other hand. For clarity, nothing in these Official Rules restricts the Sponsor or the Named Partners from using, disclosing, publishing or otherwise exploiting any ideas, suggestions or feedback provided by any Participant during the Competition for any legitimate business purpose. Each Participant further acknowledges that all products, services and other materials disclosed by any Participant during the course of the Competition are submitted on a non-confidential basis, and that the Sponsor and Named Partners will have no obligation to not disclose those items or to otherwise treat those items as confidential. Solely in the case of Participants who are not Data Science Winners, if the Sponsor or one or multiple Named Partner(s) is (are) interested in licensing or acquiring any Intellectual Property Rights or other interests in the products or services discussed in an Submission, the applicable Participant(s) will negotiate in good faith with such Sponsor or Named Partner(s) to provide such license or other interest (individually and together with other contributors, as applicable). Except pursuant to a separate written agreement with the Sponsor or one or multiple Named Partner(s), no Participant may use any trademark, brand, logo or other corporate identifier of such Sponsor, Named Partner or any of the Sponsor's or Named Partners' related entities (including, in the case of the NFL, the NFL's thirty-two professional member clubs), for any purpose whatsoever without the prior written consent of the Sponsor (in the case of the Sponsor) or applicable Named Partner(s) in each instance.

NON-EXCLUSIVE RIGHTS TO USE NAMES & ENTRY: BY PARTICIPATING IN THE COMPETITION, EACH PARTICIPANT HEREBY GRANTS THE SPONSORS THE UNLIMITED RIGHT THROUGHOUT THE WORLD TO USE, RECORD AND/OR DOCUMENT HIS, HER OR ITS (AS APPLICABLE) NAME, PHOTO, VOICE, STREAM, VIDEOTAPE, LIKENESS, STATEMENTS ATTRIBUTED TO PARTICIPANT, BIOGRAPHICAL, PROFESSIONAL AND OTHER RELATED INFORMATION IN CONNECTION WITH ANY INTERNAL OR EXTERNAL PROMOTIONAL ACTIVITIES OF THE SPONSORS (E.G., WITHOUT LIMITATION, ANNOUNCEMENTS OF FINALISTS OR WINNERS), WITHOUT COMPENSATION OF ANY KIND TO PARTICIPANT. Each Participant understands that these items may be broadcast, displayed, reproduced, stored, edited, exhibited, used and distributed by the Sponsor or the Named Partners over the Internet and/or any other communication medium now existing or hereafter created, for promotional, revenue-producing and/or any other purpose as the Sponsor or Named Partners determine in their sole and absolute discretion.

Each Participant submitting an entry to the Data Science Theme that becomes a Data Science Winner hereby grants and will grant to Sponsor, Named Partners, Kaggle and their respective designees a worldwide, non-exclusive, sub-licensable, transferable, fully paid-up, royalty-free, perpetual, irrevocable right to use, not use, reproduce, distribute, create derivative works of, publicly perform, publicly display, digitally perform, make, have made, sell, offer for sale and import their winning Data Science Submission and the source code used to generate the Data Science Submission, in any media now known or hereafter developed, for any purpose whatsoever, commercial or otherwise, without further approval by or payment to Participant; and represents that he/she/it has the unrestricted right to grant that license. Entries leveraging tools subject to the MIT, GNU GPL 3.0, and Apache 2.0 open source licenses are approved for use in the competition. Participants do not need to grant the license for software for generally commercially available software that Participant used to generate such Participant's Submission that is not owned by such Participant, but that can be procured by the Sponsor or the Named Partners without undue expense.

If the Sponsor or one or multiple Named Partner(s) is (are) interested in additional licensing or acquiring any Intellectual Property Rights or other interests in the products or services discussed in a winning Data Science Submission that are in addition to those rights set forth in the preceding paragraph, the applicable Data Science Winner(s) will negotiate in good faith with such Sponsor(s) to provide such license or other interest (individually and together with other contributors, as applicable).

9. CONFIDENTIALITY; COMPETITION DATA; SUBMISSION CODE REQUIREMENTS :

Each Participant agrees that it shall (A) use all information received from the Sponsor, the Named Partners or their respective affiliates, employees, consultants or agents that is not generally available to the public, including without limitation any injury-related information from football games (collectively, " Confidential Information "), solely for the limited purpose of creating a Submission and not for any other purpose, and (B) keep the Confidential Information strictly confidential and not provide or communicate the Confidential Information (or any part thereof) to any other person or entity without the prior written consent of the relevant Sponsor or Named Partner. Upon the Sponsor's or any Named Partner's request or upon completion of the Competition, each Participant shall promptly return (or destroy) the Confidential Information (and all copies, extracts or other reproductions made thereof) to Sponsor, in the case that Sponsor so requested, or such requesting Named Partner, or destroy such Confidential Information (at such requesting Sponsor's or Named Partner's option).

Applicable only to the Data Science Theme :

" Competition Data " means the data or datasets available from the Data Science Site for the purpose of use in the Data Science Theme, including any prototype or executable code provided on the Data Science Site. The Competition Data will contain private and public test sets. Which data belongs to which set will not be made available to Participants.

Participants in the Data Science Theme may access and use the Competition Data only for participating in the Data Science Theme and on Kaggle.com forums. The Sponsor and Named Partners reserve the right to disqualify any such Participant who uses the Competition Data other than as permitted by the Data Science Theme and these Official Rules. For the avoidance of doubt, Competition Data may include video footage from certain NFL games, and all such footage may be accessed and used only for participating in the Data Science Theme.

Each Participant in the Data Science Theme agrees:

• To use reasonable and suitable measures to prevent persons who have not formally agreed to these Official Rules from gaining access to the Competition Data;
• Not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Competition Data to any party not participating in the Data Science Theme;
• To notify Kaggle or the Sponsor and Named Partners immediately upon learning of any possible unauthorized transmission of or unauthorized access to the Competition Data and agree to work with Kaggle, the Sponsor and the Named Partners to rectify any unauthorized transmission or access;
• That participation in the Competition will not be construed as having or being granted a license (expressly, by implication, estoppel, or otherwise) under any Competition Data, nor any right of ownership in any of the Competition Data.

Participants in the Data Science Theme may use data other than the Competition Data (" External Data ") to develop and test such Participant's models and Submissions; provided, however , Participants will ensure the External Data is available to use by all Participants of the Data Science Theme for purposes of the Data Science Theme at no cost to the other Participants of the Data Science Theme.

Unless otherwise specifically permitted under the Data Science Site or these Official Rules, for purposes of participation in the Data Science Theme and from August 10, 2021 until 12:00 am on November 2, 2021, Participants in the Data Science Theme are not allowed to privately share source or executable code developed in connection with or based upon the Competition Data or other source or executable code relevant to the Data Science Theme (" Competition Code "). This prohibition includes sharing Competition Code between separate Data Science Teams, unless a Data Science Team merger occurs. Any such sharing of Competition Code is a breach of these Official Rules and may result in disqualification.

Participants in the Data Science Theme are permitted to publicly share Competition Code, provided that such public sharing does not violate the intellectual property rights of any third party. If a Participant in the Data Science Theme does choose to share Competition Code or other such code, Such Participant is required to share it on Kaggle.com on the discussion forum or kernels associated specifically with the Data Science Theme for the benefit of all competitors. By so sharing, Such Participant is deemed to have licensed the shared code under an Open Source Initiative-approved license (see www.opensource.org ) that in no event limits commercial use of such Competition Code or model containing or depending on such Competition Code.

Unless otherwise stated in these Official Rules, if open source code is used in the model to generate the a Data Science Theme Submission, then such Participant in the Data Science Theme must only use open source code licensed under an Open Source Initiative-approved license (see www.opensource.org ) that in no event limits commercial use of such code or model containing or depending on such code.

10. GENERAL CONDITIONS: In the event that the operation, security, health and safety or administration of the Competition is impaired in any way for any reason, including, but not limited to, fraud, virus, other technical problem, pandemic, epidemic or any Federal, state or local government imposed restrictions, limitations or lockdowns, the Sponsor and Named Partners may, in their sole discretion: (a) suspend the Competition to address the impairment and then resume the Competition in a manner that best conforms to the spirit of these Official Rules; (b) award the prizes from among the eligible entries received up to the time of the impairment, or (c) cancel the Competition in its entirety, in which case no prizes will be awarded. The Sponsor and Named Partners reserve the right in their sole discretion to disqualify any individual it finds to be tampering with the entry process or the operation of the Competition or to be acting in violation of these Official Rules or in an unsportsmanlike or disruptive manner. Any attempt by any person to undermine the legitimate operation of the Competition may be a violation of criminal and civil law, and, should such an attempt be made, the Sponsor and Named Partners reserve the right to seek damages from any such person to the fullest extent permitted by law. The Sponsor's or the Named Partners' failure to enforce any term of these Official Rules will not constitute a waiver of the Sponsor's or Named Partners' right to enforce such term.

11. RELEASE AND LIMITATIONS OF LIABILITY: By participating in the Competition, to the maximum extent permitted by applicable law, each Participant agrees to release and hold harmless the Sponsor, Named Partners and Sponsor-Related Persons (the " Released Parties ") from and against any liability, claims, demands, losses, damages, costs and expenses (including legal fees) arising out of participation in the Competition or receipt or use of any prize, including, but not limited to: (a) unauthorized human intervention in the Competition; (b) the processing or judging of applications and/or presentations; (c) technical errors related to computers, servers, websites, providers, or telephone or network lines; (d) printing errors; (e) lost, late, postage-due, misdirected, or undeliverable mail, or any late, lost, damaged, misdirected, incomplete, illegible, undeliverable, or destroyed Submissions or Submission materials due to system errors, failed, incomplete or garbled computer or other telecommunication transmission malfunctions, hardware or software failures of any kind, lost or unavailable network connections, typographical or system/human errors and failures, technical malfunctions of any telephone network or lines, cable connections satellite transmissions, traffic congestion on the internet or on any websites, or any combination thereof, which may limit a Participant's ability to participate; (f) errors in the administration of the Competition or the processing of entries, typographical or other error in the printing, offering or announcement of any prizes or winners; (g) injury or damage to persons or property which may be caused, directly or indirectly, in whole or in part, from Participant's participation in the Competition or receipt or use of any prize; (h) personal injury, damage or death to persons which may be caused, directly or indirectly, in whole or in part, by Participant's Data Science Theme submissions, or use thereof; (i) other errors or problems of any kind whether mechanical,human, technical, network or electronic, or errors which may occur in connection with the administration of the Competition; (j) any Submission or other material uploaded or provided by a Participant that infringes any third party proprietary rights, intellectual property rights, personal or moral rights or any other rights, including without limitation, copyright, trademark, patent, trade secret, privacy, publicity or confidentiality obligations, or defames any person; (k) any non-compliance by such Participant with these Official Rules or any applicable law or regulation; (l) claims brought by persons or entities other than the parties to these Official Rules arising from or related to such Participant's involvement with the Competition; or (m) the awarding or announcement of the winners or finalists or in any other Competition-related materials; or (n) any delay or cancellation of the Competition pursuant to these Official Rules. To the extent permitted by applicable law, each Participant waives the right to claim any damages whatsoever, including, but not limited to, punitive, consequential, direct, or indirect damages. To the extent permitted by applicable law, each Participant further agrees to defend, indemnify and hold harmless the Released Parties from and against any and all third party claims or causes of action arising out of or in any way related to Participant's product or service and the Released Parties' use thereof as authorized herein, including, without limitation, any claims of infringement of intellectual property rights.

THE SPONSORS SHALL NOT BE LIABLE TO ANY PARTICIPANT FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR PUNITIVE DAMAGES IN CONNECTION WITH THE COMPETITION OR THIS AGREEMENT. EACH PARTICIPANT HEREBY RELEASES AND AGREES TO INDEMNIFY AND HOLD HARMLESS THE SPONSORS AND THEIR EMPLOYEES, OFFICERS, AFFILIATES, RELATED PARTIES, REPRESENTATIVES, AGENTS, PARTNERS, JUDGES AND ADVERTISING AND PROMOTIONAL AGENCIES FROM ANY AND ALL DAMAGES, INJURIES, CLAIMS, CAUSES OF ACTIONS, LIABILITY OR LOSSES OF ANY KIND (INCLUDING ACTUAL LEGAL FEES AND EXPENSES), KNOWN OR UNKNOWN, ABSOLUTE OR CONTINGENT, NOW OR IN THE FUTURE ARISING FROM OR RELATED TO: (1) HIS, HER, OR ITS (AS APPLICABLE) FAILURE TO COMPLY WITH ANY OF TERMS OF THIS AGREEMENT OR OTHER APPLICABLE RULES; (2) ANY MISREPRESENTATION HE, SHE OR IT (AS APPLICABLE) MAKES TO THE SPONSORS UNDER THIS AGREEMENT OR OTHERWISE; (3) HIS, HER OR ITS (AS APPLICABLE) PARTICIPATION IN THE COMPETITION; (4) HIS, HER OR ITS (AS APPLICABLE) RECEIPT, USE OR REDEMPTION OF ANY PRIZE, OR THE INABILITY TO RECEIVE, USE OR REDEEM ANY PRIZE; (5) A CLAIM BY A THIRD PARTY THAT THE PRESENTATION, OR ANY SERVICES OR PRODUCTS DISCUSSED IN THE PRESENTATION, OR ANY OTHER MATERIALS OF ANY NATURE FURNISHED BY IT, HIM OR HER (AS APPLICABLE), INFRINGES (WHETHER DIRECTLY, CONTRIBUTORILY, OR OTHERWISE), MISAPPROPRIATES OR VIOLATES SUCH THIRD PARTY'S INTELLECTUAL PROPERTY RIGHTS; (6) A CLAIM BY A THIRD PARTY RELATED TO THE THIRD-PARTY'S USE OF PARTICIPANT'S DATA SCIENCE THEME SUBMISSION. THE SPONSOR AFFILIATES MAKE NO REPRESENTATIONS OR WARRANTIES ABOUT THE ACCURACY, RELIABILITY, COMPLETENESS OR TIMELINESS OF THE INFORMATION PROVIDED TO THE PARTICIPANT IN CONNECTION WITH THE CHALLENGE AND EXPRESSLY DISCLAIM ALL WARRANTIES, EITHER EXPRESS OR IMPLIED, STATUTORY OR OTHERWISE, RELATING TO THE CHALLENGE. IN NO EVENT SHALL THE SPONSOR AFFILIATES BE LIABLE FOR ANY DAMAGES (INCLUDING WITHOUT LIMITATION, INCIDENTAL OR CONSEQUENTIAL DAMAGES, PERSONAL INJURY/WRONGFUL DEATH, LOST PROFITS OR ANY OTHER DAMAGES) RESULTING FROM THE USE OF ANY INFORMATION PROVIDED IN CONNECTION WITH THE CHALLENGE.

12. ARBITRATION : Except where prohibited by law, as a condition of participating in this Competition, each Participant agrees that (A) any and all disputes and causes of action arising out of or connected with this Competition, or any prizes awarded, will be resolved individually, without resort to any form of class action, and exclusively by final and binding arbitration under the rules of the American Arbitration Association and held at the AAA regional office nearest the Participant's principal business location; (B) the Federal Arbitration Act will govern the interpretation, enforcement and all proceedings at such arbitration; and (C) judgment upon such arbitration award may be entered in any court having jurisdiction. Under no circumstances will Participant be permitted to obtain awards for, and Participant hereby waives all rights to claim, punitive, incidental or consequential damages, or any other damages, including attorneys' fees, other than Participant's actual out-of-pocket expenses (i.e., costs associated with participating in this Competition), and Participant further waives all rights to have damages multiplied or increased.

13. CHOICE OF LAW: Except where prohibited by law, all issues and questions concerning the construction, validity, interpretation and enforceability of these Official Rules, or the rights and obligations of Participant, the Sponsor and the Named Partners in connection with the Competition, will be governed by, and construed in accordance with, the substantive laws of the State of New York, USA. To the extent necessary to enforce the arbitration provision in Section 14 above or if such arbitration provision is not enforceable for any reason, except where prohibited by law, the Participant hereby irrevocably and unconditionally (X) submits to the exclusive jurisdiction of the appropriate federal or state court located in the State of New York, New York County, and any appellate court thereof; and (Y) agrees that all claims will be heard and determined in such court.

14. COMPETITION RESULTS: A list of the winning teams will be posted on www.nfl.com/1standfuture on or about January 2022.

15. RIGHT TO CANCEL, MODIFY OR DISQUALIFY: If for any reason the Competition is not capable of running as planned, including infection by computer virus, bugs, tampering, unauthorized intervention, fraud, technical failures, or any other causes which corrupt or affect the administration, security, fairness, integrity, or proper conduct of the Competition, the Sponsor and Named Partners reserve the right to cancel, terminate, modify or suspend the Competition. The Sponsor and Named Partners further reserve the right to disqualify any participant who tampers with the submission process or any other part of the Competition or any Competition-related website. Any attempt by a participant to deliberately damage any website, including the Competition-related website, or undermine the legitimate operation of the Competition is a violation of criminal and civil laws. Should such an attempt be made, the Sponsor and Named Partners each reserves the right to seek damages from any such participant to the fullest extent of the applicable law.

16. RIGHT TO WITHDRAW: If at such time as the identity of a Named Partner is made known a Participant no longer wishes to participate in the Competition, such Participant may withdraw from the Competition by giving written notice to Sponsor, in which case such Participant foregoes the opportunity to participate in the Competition, the related event and any prizes or other features associated with winning the Competition. Such notice may be sent to Sponsor at [email protected].

17. PERSONAL INFORMATION & PRIVACY: Each Participating Persons acknowledges and agrees that such Participating Person's name and contact information shall be collected upon registration for the Competition. The Sponsor and Named Partners may use or disclose this information and any other information contained in the Submission (A) to their affiliates, (B) as may be required to conduct and manage the Competition, including to confirm the eligibility of a Participating Person, (C) to contact such Participating Person via email for promotional purposes related to the operation of this or future competitions, and (D) to exercise the Sponsor's or Named Partners' rights under this Agreement, including any further contact of the Participating Person in connection with any commercial use of the subject of the Submission.

All data provided by Participating Persons must be accurate. Participating Persons are informed that the personal data collected in connection with the Competition will be collected by the Sponsor's or the Named Partners' servers and computer systems which are based in United States and such collection will therefore be subject to applicable United States laws. As such, Participating Persons are informed that their personal data may be transferred outside of their eligible jurisdiction (including without limitation by being transferred, stored and processed outside the European Union if a Participating Persons is a resident of the European Union) and Participating Persons' personal data may not be subject to the same requirements or restrictions relating to such personal information as may exist in the country in which it was collected. Upon request, a Participating Person will be provided with access to such Participating Person's personal data. Participating Persons also have a right to oppose the collection, storage and use of such personal data under certain circumstances. Participating Persons may exercise such right by writing to the NFL, 345 Park Avenue, New York, NY 10154. If any Participating Person is not satisfied with how the Sponsor or Named Partners addressed his or her complaint, such Participating Person may be able to raise the matter with a competent regulatory authority.

Participating Persons may also withdraw their personal data upon request; however, each requesting Participating Person will be disqualified as an entrant in the Competition prior to the determination and fulfillment of the prizes.

Each Participant and Data Science Team agrees that Kaggle may collect, store, share and otherwise use personally identifiable information provided by such Participant and Data Science Team during the Kaggle account registration process and the Competition, including but not limited to, name, mailing address, phone number, and email address (" Personal Information "). Kaggle acts as an independent controller with regard to its collection, storage, sharing, and other use of this Personal Information, and will use this Personal Information in accordance with its Privacy Policy (available at www.kaggle.com/privacy ), including for administering the Competition. As a Kaggle.com account holder, such Participant or Data Science Team has the right to request access to, review, rectification, portability or deletion of any personal data held by Kaggle about such Participant by logging into such Participant's account and/or contacting Kaggle Support at www.kaggle.com/contact .

As part of the operation of the Competition, Kaggle will transfer each Participant's Personal Information to the NFL, who acts as an independent controller with regard to this Personal Information. Kaggle will transfer such Personal Information to computer systems which are based in United States, as stated in the second paragraph of this Section 17 (and, in some cases, may therefore be transferred to a country outside the country of such Participant's residence).

IF ONE OR MORE PROVISIONS OF THESE OFFICIAL RULES ARE BE FOUND INVALID, ILLEGAL OR UNENFORCEABLE, THE VALIDITY, LEGALITY AND ENFORCEABILITY OF THE REMAINING PROVISIONS OF THESE OFFICIAL RULES WILL NOT BE AFFECTED IN ANY WAY THEREBY.

IN THE EVENT THERE IS A DISCREPANCY OR INCONSISTENCY BETWEEN DISCLOSURES OR OTHER STATEMENTS CONTAINED IN ANY COMPETITION MATERIALS AND THESE OFFICIAL RULES, THESE OFFICIAL RULES WILL PREVAIL, GOVERN AND CONTROL.

1st Place - $50,000

2nd Place - $25,000

3rd Place - $13,000

4th Place - $7,000

5th Place - $5,000

Evaluation Timeline Prizes Code Requirements

Submissions are evaluated using a weighted accuracy metric for all active players’ helmets in the provided videos. Helmet boxes associated with a definitive impact will be weighted 1000x more than helmets not involved in an impact. Ground truth helmet boxes will be paired to the submission box that has the the largest Intersection over Union IoU from the submission. The IoU between the ground truth box and submission box must meet a minimum IoU threshold of 0.35. The IoU of a proposed bounding box and a ground truth bounding box is calculated as:

IoU ( A , B )= A ∩ BA ∪ B

The metric will assign each ground truth helmet to a single submitted box based on IoU.

Accuracy will be calculated for all ground truth helmet boxes excluding sideline players (labels H00 and V00). Helmet boxes are counted once for each frame they appear in a video. Helmet boxes at the moment of a definitive impact are given a weight of 1000. All other helmet boxes are given a weight of 1.

WeightedAccuracy = TotalCorrectnonimp +( TotalCorrectimp ∗1000) TotalHelmetsnonimp +( TotalHelmetsimp ∗1000)

• TotalCorrect_nonimp is the number of correctly assigned non-definitive impact helmet boxes.
• TotalCorrect_imp is the number of correctly assigned definitive impact helmet boxes.
• TotalHelmets_nonimp is the total number of non-definitive helmets boxes.
• TotalHelmets_imp is the total number of definitive helmet impact boxes.

Submissions must also meet the following requirements:

• No more than 22 helmet predictions per video frame. Predictions should not be submitted for players not actively participating in the play who are on the sideline. Sideline players are labeled “H00” and “V00” in the training dataset.
• A players’ helmet label must only be predicted once per video frame, i.e. no duplicated labels per frame.
• All submitted helmet boxes must be unique per video frame.

Submission File

For each video_frame in the test set, you must predict a bounding box left, width, top, height, and predicted label of the assigned player. The file should contain a header and have the following format:

video_frame,left,width,top,height,label 57590_003607_Endzone_1,1,1,1,1,H1 57590_003607_Sideline_1,1,1,1,1,V59 57595_001252_Endzone_1,1,1,1,1,V52

CODE REQUIREMENTS

This is a Code Competition

Submissions to this competition must be made through Notebooks. In order for the "Submit" button to be active after a commit, the following conditions must be met:

• CPU Notebook <= 9 hours run-time
• GPU Notebook <= 9 hours run-time
• Internet access disabled
• Freely & publicly available external data is allowed, including pre-trained models
• Submission file must be named submission.csv

Please see the Code Competition FAQ for more information on how to submit. And review the code debugging doc if you are encountering submission errors.

Other Challenges You May Be Interested In

• partner with us
• cookie policy
• organizations involved
• events & webinars
• general forums
• how HeroX works
• knowledge base
• innovation resources

Machine Learning

Our Solution for the NFL Helmet Assignment Kaggle Competition

Kaggle competitions attract thousands of data science and machine learning enthusiasts by providing access to various datasets and infrastructure with GPU availability and discussion forums to share ideas; this is very important for practitioners looking for a starting point.

Here at XmartLabs, we're always looking for ways to contribute to real problems that have a genuine impact on people. Kaggle is an excellent example of this because it gives organizations a platform to present their issues so that savvy technical people can solve them.

For this competition, the NFL wanted to find a solution to a critical problem they have: Player's Health & Safety . This is not the first time they have cooperated with Kaggle; they set out to identify helmet impacts in a previous competition. This time around, they wanted to determine which helmet belongs to each player and keep track of their collisions through a match. Currently, the tracking is done by hand, but an automated algorithm would significantly reduce the time needed to do it manually, freeing more resources to do complex research about how to ensure players' safety in the field. In this  blog post , the NFL gives its statement on the importance of this competition.

The Challenge

The dataset has two sources of information, the videos, and the tracking information. The objective is to match the detected helmets in each video frame to the label of its corresponding player, which are identified by their shirt number and whether they're in the home or visitor team; some examples of players' labels are 'H67' or 'V9'.

The video training datasets consist of 60 short plays (around 10 seconds videos with 60 fps). Each play is filmed from two synchronized cameras, one located at the endzone and the other one at the sideline, concluding in a total of 120 videos in the training dataset.

Left image: SidelinView. Right image: Endzone view.

The tracking information comes from a device located in the player's shoulder pad. It gives the relative position of each player on the field with other information about the player's movement at a 10Hz frequency. The tracking has the 'x' and 'y' positions described in the image below.

Other supplementary information given in tracking is speed, acceleration, distance traveled from the last point, and orientation. Below, you can see an example of the player's position at the beginning of a game.

Another file given at the start of the competition was the location of the helmets in a given frame, predicted by a baseline model. These predictions are not perfect; the reason for giving them is to provide competitors with a starting point independently from the detection of helmets to focus on the real problem. The training images are given so you can train your own helmet detector, and although we didn't take this approach, some top-scoring teams did. There's an interesting  discussion  around this topic because top-scoring competitors tried the same solution but with the baseline helmet detector and found a 10% decrease in the score.

Our Solution

Our solution is made of many sequential modules, and each of them tackles a specific task. These modules can be worked on individually, allowing us to develop and fix some of them in parallel. We order the modules by importance and complexity:

1. 2D Matching:

The central part of this problem is mapping the helmet's positions in a given frame with the tracking coordinates. As both sources of information are in a two-dimensional space, the mapping has to be done accordingly; what we call '2D Matching'. As simple as it can be for a human to match two clouds of points (with reasonable similarity), there's more than one way to do it automatically on a computer. Various algorithms tried to solve this optimization problem, where the value to optimize is the distance between the matched pairs.

In our case, we used the method proposed in the paper "Robust Point Set Registration Using Gaussian Mixture Models," which has its  code  available in Python. With some minor changes, we were able to include it in our solution.

Left: Example shown in code presentation, Right: Example of implementation in our solution.

When passed two normalized clouds of points, the algorithm can return a correspondence between those cloud points. There are some issues; for example, when both sets have 22 points, the algorithm works really well, but when the video has a frame with fewer helmets, the results might have some errors. Because of this, we decided only to do 2D matching when there are 15 helmets or more in the image frame. Later we'll explain how we propagate the labels to the frames that don't have enough helmets. Besides this, we didn't run the matching on all frames, as variations in consecutive frames were almost none.

Example of a low number of helmets in the frame.

2. DeepSORT:

Once the matching gets done, we transfer these intermediate results to the  DeepSORT  module. By using DeepSORT, we add a temporal dependency between the frames of a video. This algorithm is commonly used on object tracking problems; it tries to keep up with a helmet throughout the video by assigning it an ID. Finally, based on the 2D matching results, it looks for the most given label for that helmet and designates it as its final label.

We used an idea to match only in frames we were confident the results would be good. By doing this, DeepSORT took only those certain frames' labels and spread them throughout the video.

3. DeepSORT Correction:

Our output from deepSORT did have errors, so we wanted to come up with a creative solution. One of the main issues were duplicated labels and 'None' labels (when no valid labels were assigned to that helmet). These errors didn't appear on all frames, so we used the frames that had all the helmets matched to correct the other ones. We decided to do a two-step solution, first fixing simple duplicated errors and more significant problems after.

The first step addressed cases where there were only two identifiable errors, a duplicated pair. Assuming that all the other labels were correct, we decided to pick the four closest helmets to one of the duplicates. Using a homography matrix, we transferred the duplicate to the tracking space and found the closest player there. By analyzing the results of each helmet in the pair, we built an algorithm to decide how to correct it.

After fixing those simple errors, we matched each remaining frame with detectable errors and the nearest one without errors. In this case, we used the  Hungarian Algorithm  to perform the matching.

In some cases, this implementation gave impressive results. As seen in the following video, green bounding boxes are correctly labeled helmets and yellow when there is an impact (higher reward for the correct label while collision). The red bounding boxes are due to wrong labels during a collision.

Output of our solution applied to a Sideline video.

4. Image Rotation:

The initial matching was done by doing many iterations looking for the best rotation angle that could fit the helmets in the x-axis with the tracking data. This is very expensive computationally, so we decided to help out the algorithm by finding the best rotation without iterating. After changing the matching, this module was kept since it also helped the '2D-matching'.

Our solution detects the field lines using  the Canny edge detector  algorithm followed by  the Hough line transform . With those detected lines, the rotation angle is calculated. By rotating, we can position the helmets in a way that's more representative of reality.

As seen in the example of the two marked players, the bottom one appears to be more to the right; however, in reality, the top one is further to the right. By doing the rotation, the problem is solved.

The rotation is done using the purple line, and the angle with the green one is the rotation angle.

Image rotation example.

5. Outfield players:

Another issue we saw and addressed was the presence of off-field players/helmets. The baseline helmet detector didn't discriminate between these two types of helmets, so some labels were assigned to outfield players. This meant those labels weren't being assigned to the correct player, which generated an offset that moved all labels to another helmet.

To solve this, we detected sideline lines to differentiate these two types of helmets. One difference with image rotation is that, in this case, we looked for wider lines, like the one shown in the image.

Most of the detected helmets are field players, so considering all helmets as players gives a precision of 0.983. With our detector, precision was improved to 0.994.

6. Camera angle detection:

Each video was labeled "Sideline" or "Endzone," but it wasn't specified from which side or field end it was being recorded. There were two possibilities for each. Identifying this quickly and confidently can signify a correction before the matching, which facilitates it and means it needs fewer iterations.

Left: Endzone example. Right: Sideline example.

At the beginning of each video, we saw that the formations were very identifiable and easy to correlate with the tracking. In other words, with the correct view, the 2D matching should have a high rate of matches. So the implementation takes five frames distributed in the first 100 frames. In these selected frames, the 2D matching is done for each possible case; the one with the most matches is selected as correct; after doing it for all five frames, the correct side is selected and saved.

Ideas that didn't make it into the final solution

For different reasons, not everything we tried made it into the final solution. Here we present two of the ideas that didn't.

Team clustering is the idea of separating the helmets by each team. The idea was that the 2D matching algorithm worked better if it had to match 11 helmets with 11 labels twice instead of 22 with 22 once. Our first idea was to use the helmet's color as the colors are very different most of the time. We run tests trying three color spaces:  RGB ,  HSV , and  CIELAB .

The use of HSV and CIELAB is to get values independent from illumination and other environmental phenomena. Some top scoring solutions did focus on team clustering but with different approaches, for example, using deep learning models.

Although cameras were fixed in a position, they still could rotate and zoom in or out during a play. We tried to capture this information to leverage it in the solution because if we know how the image is moving, we can predict which players are leaving the frame and discard their tracking information. Next, we look at the difference between the first frame of a video and the last.

Calculating the size and number of the helmets in the frame, we can have a good approximation of how the zoom is going:

However, merging this information with our pipeline wasn't easy, as more processing had to be done, and at the time, it didn't seem relevant.

Solutions from other competitors

As the competition came to a close, scores on the leaderboard started to soar. Top teams got our attention, and we were looking forward to the end of the competition to see what they had done. Showing your solution and code is optional, but it is common for top Kaggle contestants to share their work.

Looking at other teams, we can confidently say that our ideas weren't that far from top-scoring solutions. In our opinion, the most creative solution was the one that came on top ( notebook  and  explanation  are public). On this code, as in many others, a helmet detector was trained. Most of these detectors are  YoloV5  architectures trained on the competition's data. However, this one uses a 2-stage detector, whereas the first model resizes the images, and the second one is trained to predict this resized image.

From here on, the code gets more complex, but to keep it simple, we can synthesize the rest of the code as follows:

• Image to Map converter : A Convolutional Neural Network (based on the  U-Net ) thats' trained to transform the detected helmet's positions into the tracking space.
• Team Classification : With another CNN, the helmets in a frame are processed, and after that, separated into two teams. This classification doesn't predict if the teams are Home or Visitors.
• Points to Points Registration:  Using  Iterative Closest Points , the best transformation is selected to minimize the distances between both cloud points.
• Tracker : Using  IoU  tracker, it accumulates the results of player assignment through all frames and re-assigns players to bounding boxes.
• Ensemble : Applying  ensemble learning  with four models, the final prediction benefits from multiple "opinions."

Other solutions are more ground-based and share points with ours. Looking at them made us realize where we had to emphasize and where we had put too much work.

We want to thank the Kaggle hosts and the NFL for this fantastic competition. Although our solution was among the top 50, we would have loved to prove ourselves further. However, we still had fun trying new stuff and learning a lot along the way. Kaggle is a great place to work on your ML skills, and the community is always open to giving a helping hand. We know that we will do much better the next time by applying what we learned from this experience about competing in a Kaggle competition (you can read about it  here ).

ML Kit on iOS and how it performs against Core ML

How to convert a NN model from TensorFlow Lite to CoreML

5 Practical Ways to Speed Up your Deep Learning Model

What are guardian caps? What to know about NFL's new helmet initiative

Players can start wearing the caps in the regular season beginning in 2024., by sanjesh singh • published april 26, 2024 • updated on april 26, 2024 at 7:57 pm.

New helmet protection is coming to the National Football League .

The NFL announced on Friday it authorized guardian caps to be used during regular-season games, a first for the league.

To reduce player risk and continue driving innovations in equipment technology, Guardian Caps have been authorized for in-game use this season. Hear more from @NFL Chief Medical Officer, Dr. Allen Sills: https://t.co/PpqOBxEiHl pic.twitter.com/tXUlAaEDbR — NFL Football Operations (@NFLFootballOps) April 26, 2024

Here's what to know about guardian caps entering the 2024 NFL regular season:

Get Tri-state area news and weather forecasts to your inbox. Sign up for NBC New York newsletters.

Roger Goodell considering 18-game NFL regular season and Super Bowl on Presidents Day weekend

Why Caleb Williams paints his nails

What are guardian caps.

Guardian caps are soft shell covers on top of helmets that provide extra cushioning and enhanced safety for head-to-head contact.

When did the NFL start using guardian caps?

The NFL started allowing guardian-cap usage during the 2022 training camp before making it mandated for preseason practices in 2023. That then rolled over to regular-season and postseason practices.

Are guardian caps required for NFL players?

Guardian caps have been required for certain positional groups during practices. Those include lineman, linebackers, tight ends running backs and fullbacks since head-to-head contact is more frequent among those positions. But that list is expected to expand to wide receivers and defensive backs.

Quarterbacks, punter and kickers are not required to wear them in practice.

Guardian caps will be optional for any player in the regular season.

What are the impacts of wearing a guardian cap?

Guardian caps are used to help lower the rate of head-contact injuries. The caps can reduce the force from head contact by 10% if one player is wearing it. That number rises to 20% if all players involved are wearing them, according to the NFL when the caps debuted in 2022.

This article tagged under:

NFL will allow Guardian Caps on helmets during games for 2024 season

The NFL will allow players to wear Guardian Caps — soft-shell covers used on helmets — in games this upcoming season, the league announced.

“There is the option for a player to wear it in a game if he so chooses,” Dawn Aponte, the NFL’s chief football administration officer, said in a health and safety webinar Tuesday .

Advertisement

As part of its effort to focus on player safety and reducing head injuries, the NFL began to mandate the use of Guardian Caps for players who play certain position groups during training camp practices in 2022. According to the league, it resulted in a 50 percent decrease in concussions among players who used Guardian Caps.

To reduce player risk and continue driving innovations in equipment technology, Guardian Caps have been authorized for in-game use this season. Hear more from @NFL Chief Medical Officer, Dr. Allen Sills: https://t.co/PpqOBxEiHl pic.twitter.com/tXUlAaEDbR — NFL Football Operations (@NFLFootballOps) April 26, 2024

In response, the NFL extended the mandate to regular-season practices last year. For next season, the league will expand the list of position groups required to use Guardian Caps to running backs, fullbacks, tight ends, receivers, offensive linemen, defensive linemen, linebackers and defensive backs. The use of Guardian Caps remains optional for quarterbacks, kickers and punters.

There have been players who have complained about the Guardian Caps having an awkward fit, but NFL officials believe they’ve become more accepted.

Free, daily NFL updates direct to your inbox. Sign up

“I think there was a lot more acceptance this year; you heard less pushback.” NFL chief medical officer Dr. Allen Sills said in February. “One of the things to remember is an overwhelmingly large number of Power 5 football teams use Guardian caps already. And so many of, if not most of, the players coming to the league are used to it. So, it’s not as big of a transition for them as it is for veteran players.

“We’ve seen players be motivated by the data and that’s why it’s important that we share it and we do this work collaboratively with the (NFL players’) union.”

In aiming to reduce in-game concussions, the NFL has focused on position-specific helmets, with eight new position-specific helmet models to be introduced this season.

“The growing availability of position-specific helmets is a central part of our effort to provide players with the best possible protection for their position,” Sills said in February. “Quarterbacks and linemen will have a range of tailored helmets available to them with more positions to come in the next couple of years.”

While the use of Guardian Caps in NFL practices has steadily increased in the last couple of years, they were never permitted for use in preseason, regular season or postseason games until now. According to the NFL, Guardian Caps will absorb at least 10 percent of the force when a player wearing one endures a helmet-to-helmet hit. And, if both players are wearing Guardian Caps during a helmet-to-helmet hit, the force of the impact is reduced by at least 20 percent.

The hope is that the potential use of Guardian Caps in games, rule changes such as the revamped kickoff format and helmet advancements will help reduce the number of in-game concussions. The total number of concussions suffered in practices and games in the NFL has increased every year since 2022.

“We want to see them go down,” Sills said. “We’re not satisfied with the number where we are and we believe the game can be continued to be made safer.”

Required reading

• NFL reports lower-body injuries down, concussions up in 2023 season

(Photo: Kirby Lee / USA Today)

Get all-access to exclusive stories.

Subscribe to The Athletic for in-depth coverage of your favorite players, teams, leagues and clubs. Try a week on us.

Kaggle: NFL Helmet Assignment(Top8%铜牌区)解决方案及前排高分思路

原文阅读请关注微信公众号： 宅码

NFL赛程3个月...导致鸽了这么久。这是今年目前我参加的最有意思的比赛-Kaggle: NFL Health & Safety - Helmet Assignment[1]，比赛打完仍意犹未尽，特此分享我们团队的解决方案与高分选手的思路（有对比才有伤害），不足之处，还望批评指正。

【背景】美国国家橄榄球联盟 (NFL) 和亚马逊网络服务 (AWS) 正在合作开发最佳的运动损伤监测和缓解计划。去年Kaggle NFL比赛是做头盔目标检测，今年难度升级。在今年的比赛中，赛方希望选手能从视频片段中识别和分配足球运动员的头盔。特别是，构建能够通过跟踪信息（Tracking Dataset，即NGS信息）将检测到的头盔分配给正确玩家的算法。我先介绍下NGS追踪数据是什么？NFL 的 Next Gen Stats (NGS) 程序采用复杂的跟踪技术通过每个球员肩垫中的RFID设备和嵌在每个体育场中的设备收集数据。这些设备会捕获多种数据，包括特定时间内上场球员名单、精准到英尺的球员位置、球员移动速度和方向等。[2]，NGS追踪数据如下图所示:

图1：NGS追踪数据样例图

由于追踪数据是在每个球员垫肩上装设备采集到的数据，所以图1中每个点可以知道对应的球员ID是什么，而赛方希望我们做的事情是，基于追踪数据，给下面视频画面中的球员头盔匹配上正确的球员ID（注意：每场NFL比赛，有两个视角录制的视频）：

图2：NFL视频画面样例图

【数据】关于数据集，总结有以下几类：

• images文件夹内的jpg文件和image_labels.csv：补充数据，可用于自行训练头盔目标检测器。
• [train/test]_baseline_helmets.csv：若不想头盔检测，可以直接用官方给的头盔检测baseline模型（仅用images和image_labels训练）的检测结果，包含vedio_frame下bbox和conf。
• [train/test]文件夹内的mp4文件：记录每场play，每个play有两份: Endzone和Sideline (两个视角)。这两份视频每帧能按时间匹配上。但在不同视角下，可能不同球手的可见程度不一样，你只需要预测真实可见的选手即可。
• [train/test]_player_tracking.csv：NGS追踪数据，每个球员都佩戴一个传感器，让我们可以在场上精确定位他们。
• train_labels.csv：训练集中，头盔追踪和碰撞标签。

【评估指标】由于赛方更关注头盔撞击的bbox有无正确分配到正确的球员ID，评估函数对撞击头盔的检测结果做加权，权重是无撞击头盔的1000倍。

Weighted Accuracy = \frac{TotalCorrect_{nonimp}+(TotalCorrect_{imp} * 1000)}{TotalHelmets_{nonimp}+(TotalHelmets_{imp} * 1000)}

【时间】2021年8月16日-2021年11月2日(近3个月)。

【排名】48(Public)/64(Private)，总共825支队伍，Top8%铜牌区。

【代码】 https:// github.com/AlvinAi96/ka ggle-NLF-helmet-assignment

【团队】团队Baseline Plus Pro共三人：

• Shao Eric: 邵兄，江南大学研究生，研究方向: 知识蒸馏，知乎id: 闪电侠的右手。
• Leonrain: 刘兄，德国软件工程师，工作方向: 目标检测。
• Alvin.ai: 我，ML研究员，工作方向：机器学习。

我们整体方案流程图如下所示：

图3：团队Baseline Plus Pro的整体解决方案流程图

我们有两个视角，分别是Sideline和Endzone，而每个视频可能处于不同侧，例如图1，Sideline的视频可能是在Home或Visitor一侧录制的，而Endzone也一样。因为后面要基于球员位置，将追踪数据中的球员ID匹配到视频头盔上，所以搞清楚当前视频下的摄像机在主/客队哪一侧是很有帮助的。我们针对Sideline和Endzone使用并尝试了不同的机位预测方法。

(1) Sideline机位预测

我们尝试了三种Sideline机位预测方法：

(a) x mapping ：基于@tito开源的思路[3]，分别对视频首帧画面头盔位置x和追踪数据中球员位置x进行的距离排序，然后根据排序结果，依次匹配球员，计算匹配距离。若视频画面180旋转后，与追踪数据的匹配距离最小，说明当前机位为Visitor Sideline，而不是Home Sideline。这也是我们最终采用的方法，线下准确率：59/60，错误的视频是存在某个场外替补球员没法很好的剔除，导致破坏了匹配位置的对应性。

图4：基于x位置的Sideline机位预测

(b) 基于地标线斜率和球员分布 ：这是@Adrian 开源的方法[4]，首先，我们得留意一个背景：NFL比赛场上主客队上场球员各自是11个，可实际上，由于摄像机的位置和操作（旋转/位移/缩放）且头盔存在遮挡，导致实际头盔目标检测后的头盔数量，有三种情况：

• 视频画面上头盔检测数>22个：多数是由于场外替补球员的头盔/观众的头或帽子被检测出来，少数是由于场内球员头盔被过度检测了。
• 视频画面上头盔检测数=22个：绝大多数是画面涵盖了所有场内22个球员。
• 视频画面上头盔检测数<22个：摄像机放大视角，记录的局部画面内球员数较少。

针对头盔检测数>22个的Sideline场景，Muresan使用了球员中心点和地标线斜率，具体步骤如下，流程可见图5：

• 检测出第4帧视频画面中的白色地标线（BGR转灰阶→Blur→Canny边缘检测→HoughLinesP检测直线端点）。
• 根据地标线端点计算斜率slope，以便知道摄像机朝向。
• 计算追踪数据中，所有球员位置x的平均值x_center，以便知道球员聚集的中心位置。
• 若(x_center-60)*slope<0，说明机位为Home Sideline，否则为Endzone Sideline。

图5：基于地标线斜率和球员中心点的Sideline机位预测

针对头盔检测数≤22个，则用球员在xy轴上的偏度skew，具体步骤如下，流程可见图6：

• 计算视频画面内，球员在x和y轴的偏度（sk_x_he和sk_y_he）。
• 计算追踪数据内，球员在x和y轴的偏度（sk_x_tr和sk_y_he）。
• 若sk_x_he和sk_x_tr同sign，且sk_y_he和sk_y_he同sign，说明机位为Home Sideline，否则为Endzone Sideline。

图6：基于xy轴偏度的Sideline机位预测

当时我用Muresan的方案，在线下测试，准确率100%，但用在线上发现分数掉的很厉害，后来在评论区问了他，他的回复我说：“If the video contains many helmets on the sides, outside the playing field, I expect trouble to show up.”，对此，我们团队后面有做场外球员/观众的剔除（具体方法放后面讲）。

(c) 基于球衣颜色 ：我们观察到，主队一般都是白色球衣，客队是深色球衣。然后想着基于球衣颜色直方图做机位预测，具体步骤如下，流程可见图7：

• 对视频画面球员进行目标检测，获取球员bbox后，对绿色背景mask掉后，计算颜色直方图。
• 基于各球员颜色直方图做Kmeans聚类，分为2类，获取各自的中心点直方图。
• 计算各队中心点直方图与纯白色直方图的cosine距离，距离的中心点认为是主队。
• 对比追踪数据球队的位置，判断Sideline机位。

图7：基于球衣颜色的Sideline机位预测

这个方法预测出来结果很糟糕，因为直方图提取的颜色特征不具备明显的区分度，导致预测不确定性很大。

(2) Endzone机位预测

Endzone机位预测时基于OCR识别出的球衣号，在追踪数据中哪一侧，由此判断摄像机位置。如图8所示。比如我对首帧画面检测到26号和2号（一般来说，背对摄像机的球员被检测出号码的可能性更大），然后在追踪数据里找26号和2号的位置，发现他们更靠近Visitor Endzone，因此判断视频是在客场Endzone下拍摄的。但有时候首帧画面OCR是检测不出号码的（不知道为什么，肉眼是很清晰的，可能是OCR需要fine tune吧），这时候我们允许它一直搜索到80帧（太后面的帧里球员会抱在一团，机位预测风险很大，不具有位置参考性），准确率能达到54/60，效果远没有Sideline的好，机位预测难度较大。

图8：基于OCR检测的球衣号码的Endzone机位预测

注意，机位预测的好处在于减少后面匹配的搜索空间，坏处在于，一旦预测错误，对匹配的影响也是致命的。

前面也提到，会出现如下图所示的球员冗余问题，多数是由于场外替补球员和观众被误检导致的。为了解决这个问题，我们采用了边框裁剪冗余球员的方法，主要是对画面边缘的bbox不断剔除，直到场内球员数是在一个合理范围内。这样增大了我们后面匹配的正确性搜索范围。 边缘最多允许裁剪4轮，每轮裁剪宽度为40pixel。

图9：裁剪画面边缘冗余bbox

即使是剔除了边框冗余球员，场内也会因为检测到裁判、头盔误检和摄像机视角缩放，导致视频画面的球员头盔数与追踪数据的球员数对不齐。对于这种情况，我们会有以下处理方法：

(a) 视频球员数>追踪球员数：做边框裁剪完，基于头盔检测置信度，优先剔除低置信度头盔的球员。

(a) 视频球员数<追踪球员数：若视频中球员数有20个，而追踪球员有22个，那么从22个追踪球员中遍历出不同球员的组合（每个组合只有2个球员），然后从追踪球员中删除不同组合内的2个球员，计算视频和NGS的球员的匹配距离，取匹配距离最小的采样组合，然后将其从22个追踪球员中删除。但问题在于视频缺失球员越多，需要遍历搜索的组合空间就越大，为此，设置了搜索组合的次数上限为2000次。

由于不同机位姿态导致视频画面属于鸟瞰图，而非俯视图，这样导致与NGS标准图的球员位置存在扭曲。为此，我们对视频头盔点经过旋转矩阵变换后，再与NGS图做球员匹配。图10展示了球员旋转的流程，我们设置的最小旋转角为-30，最大旋转角为30，然后间隔3个角度，抽取一个旋转角度旋转球员位置，最后选择能使后续球员匹配总体距离最小的旋转角度。

图10：利用旋转矩阵旋转球员

(4) xy Mapping

我们分别针对球员在x轴和y轴的位置分别做球员ID匹配，然后选择整体匹配距离最小的匹配结果，如下图所示：

图11：xy Mapping

前面的工作在于匹配上球员ID，现在我们使用DeepSORT[5]对球员进行跨帧的追踪，DeepSORT是2017年提出的一种多目标跟踪算法，相比于SORT，它提取了目标的外观特征放入最近邻匹配中，能有效提高目标追踪的效果，也减少ID Switch问题的发生。由于追踪目标时需要度量目标距离，Deepsort便提供了两种距离度量方式[6]：

• 马氏距离：适用于运动不确定性很低的场景，在图像空间中使用kalman滤波进行运动状态估计只是一个比较粗糙的预测。特别是相机存在运动时会使得马氏距离的关联方法失效，造成出现ID switch的现象。作者lambda=0即不使用马氏距离作为最终的度量方式。
• 余弦距离：作者对每一个追踪目标构建一个gallary，存储每一个追踪目标成功关联的最近100帧的特征向量。那么第二种度量方式就是计算第i个追踪器的最近100个成功关联的特征集与当前帧第j个检测结果的特征向量间的最小余弦距离。

距离度量(M distance)对短期的预测和匹配效果很好，但对于长时间的遮挡的情况，使用外观特征的度量(cosine distance)比较有效。对于存在相机运动的情况，作者建议置λ=0。当一个目标长时间被遮挡之后，kalman滤波（想要深入了解的朋友建议阅读： 卡尔曼滤波 ）预测的不确定性就会大大增加，状态空间内的可观察性就会大大降低。假如此时两个追踪器竞争同一个检测结果的匹配权，往往遮挡时间较长的那条轨迹因为长时间未更新位置信息，追踪预测位置的不确定性更大，即协方差会更大，马氏距离计算时使用了协方差的倒数，因此马氏距离会更小，因此使得检测结果更可能和遮挡时间较长的那条轨迹相关联，这种不理想的效果往往会破坏追踪的持续性。总结来说，KF碰到目标长时间遮挡问题，会“放弃”追踪后面遮挡完出来的目标，因此作者设lambda为0，即不考虑马氏距离的度量。

回到我们的赛题，我们会发现，如果只是追踪头盔bbox的外观内容，是非常不友好的，因为头盔遮挡住面部特征，而且同一支球队的头盔外观是一样的，那DeepSORT基于外观特征的相似度做度量，是存在很大风险的。本来我们是想去修改源代码中lambda参数的设置（增加lambda），但发现作者开源代码中并没有把lambda给实现出来，而是直接忽略了它。为此，我们除了基于头盔度量，还对球员bbox（通过头盔bbox外扩得到）做DeepSORT追踪，因为相比于头盔bbox，球员bbox更适配Deepsort的预训练好的行人特征抽取器，同时球员的体态特征或球衣号特征可能也会被捕捉到。但我们对比实验后，发现，有些部分视频下，头盔DeepSORT结果更好，部分视频是球员DeepSORT结果更好，但所有视频的总体评分是头盔DeepSORT更高一些。猜测原因是：相比于球员，头盔被遮挡的可能性更小，增大目标暴露的可能，所以某些时候比球员DeepSORT好。

为了平衡头盔DeepSORT和球员DeepSORT的结果，我们做了融合，当某球员的label分组计数*0.6还大于头盔的label分组计数，我们认为信赖该球员的球员追踪结果，而不是头盔追踪结果。对了，我们对Sideline和EndZone还分别调了对应的两套参数。

前面其实已经输出了很多额外尝试的内容，这里补充下。

我们中途尝试过使用YoloV5训练头盔检测器，而不是用官方提供的baseline头盔模型检测结果，但在线上提交代码运行报错，后面发现存在视频画面里头盔过于小，而检测不出来，导致无头盔bbox匹配而报错。后面对比高分方案，发现是自己训练时图片尺寸设为736，导致模型对小目标检测难度很大（我的锅），所以就没获取到这部分的收益。

在球员采样部分，我们尝试局限采样范围为中心球员周边范围内，但发现效果不好，原因可能是：场内中间球员存在多检测的头盔bbox，如果保留它们去做匹配，会破坏匹配的连续性。在xy Mapping上，我们尝试过基于头盔（x，y）的几何距离匹配，但速度会受影响，就没用上。还有我们还试过Passos的点云法[7]，使用梯度下降去收敛转换矩阵H，其实线下拿单个测试，效果是比上面提到用旋转矩阵的方法好，可惜即使后面我们优化成batch操作，遍历各帧做梯度下降还是速度太慢。后面看了前排的思路，也叫ICP(Iterative Closest Points, 迭代最近点)算法，他们采用了一些方法解决的速度上的问题（后面会讲）。

追踪模型选择DeepSORT是因为主持人开源了它的baseline，上手很容易。后面也有选手开源了ByteTrack，但效果没有提升。我们也试了FairMOT，但是因为线上运行代码不允许联网，导致所有依赖得本地装好，打包成数据集上传调用才行，辛苦刘兄在FairMOT上搞依赖搞了半天，特别是线上如何编译DCNv2上，花了好多心力，但由于FariMOT是将检测和跟踪进行了端对端的训练，我们这边便来不及上线调整代码架构，便没用上。

压轴的终于来了，我跟队友花了3小时复盘了前排高分思路，收获很大。接下来，你能欣赏到K神的杰作，他的方案完整性高，发paper应该问题不大。还有老牌选手猫哥的方案，相比于K神的巧夺天工的学术气息，猫哥的方案工程气息更重，有做拿简单的操作收获最大利益，读来有点大道至简的感觉。希望对大家有所启发。

1. 第1名[8]（@K_mat）

K_mat的pipeline（Detector → Converter → Classifier → Registration → Tracker）如下：

• 训练头盔Detector；
• 使用Converter将视频图像映射到2D图上(鸟瞰视角)；
• 训练Classifier去分类球员2团队（主客队H/V）；
• 对在2D地图上被检测的球员，注册（Registration）到提供的跟踪数据上；
• 跟踪检测bbox和球员重分配。

K_mat认为他与其它选手拉开距离的地方在于mapping和registration模块。

图12：第一名方案pipeline

(1) 头盔Detector

采用了2阶检测器寻找头盔：

第1阶检测器：用EfficientNetV2s检测头盔，计算被检测出的头盔的平均尺寸，然后基于头盔平均尺寸，缩放图片让头盔尺寸在25*25附近。

第2阶检测器：在第1阶检测器输出的高分辨率图片上再检测头盔（作者认为检测相同尺寸的头盔比检测不同尺寸的头盔更容易）。

图13：K_mat的二阶Detector思路

读后感：这个思路很赞，就像前面提到，我们也做过Yolov5头盔检测器，想要替换官方的头盔检测结果，但后面发现存在个别视频的机位拉的太远，导致头盔变成极小目标，增大头盔漏检风险，最后提交分数很不理想。而该思路结合业务特性设计了基于检测头盔bbox去调整图片尺寸的操作，进而减少第2个检测器的检测难度，从而解决了这个问题（这不比费劲心思为小目标调anchor box size、搞多尺度训练和预测等香吗？）

(2) Image2Map Converter

• 使用CNN（基于U-Net）将图片中的头盔bbox转为2D map。它预测在相机位置下球员的2D位置（x和y），可以理解为NGS Map上的位置。
• 它从bottleneck输出全局位置，从decoder输出小的残差。
• 头盔检测框的attention提高了准确性。

图14：Image2Map Converter

读后感 ： GT是NGS坐标。Pred是视频坐标，设计损失函数去使GT和pred点匹配距离最小。关于delta x/y和global x/y，作者绘图解释如下所示，简单来说global坐标是球员在NGS的坐标，delta x/y是球员头盔bbox中心点和垫肩传感器采集位置的相对距离偏差。

图15：球员绝对位置和相对位置

(3) 点对点Registration

将2D Map上预测的球员匹配到提供的跟踪数据上（Tracking Data）：

• 以ICP(Iterative Closest Points, 迭代最近点)算法为基础，通过最小二乘迭代拟合，求解最近邻搜索和正规方程，得到4个未知参数的解（xy平移距离、旋转角度和缩放比例）。
• 使用预/后处理删除不合适的Sideline球员。

图16：Points2Points Registration

读后感：注意这里的随机初始化很重要，因为不同初始化话去做ICP效果更好，而之前我们没做图像变化后去直接做ICP，导致匹配过于局限，影响分数。而且每次初始化，如果跑几轮发现ICP错误很大，就立刻止损，减少了不必要搜素的可能性。

图17：Points2Points Registration的后处理

读后感：之前我们尝试开源的点云代码时，面临的问题就是参数搜索的时间成本太大，但由于点云效果很不错，想着说每帧参数搜索前，先拿历史帧的搜索结果做初始化，这样迭代收敛速度会不断加快，虽然最后嫌麻烦没实现，但看到K_mat的方案里类似的做法，还是比较开心自己有这样的想法的。

(4) 队伍Classifier

队伍信息对提高Registration准确性很重要。XY位置和队伍可以在点对点Registraion中结合。CNN分类器预测出相似矩阵去表示每一对球员是否为同队。使用arcface和伪标签可以提高准确性，验证集分数大约在97%左右。

图18：队伍Classifier

读后感：对比我基于球员mask的颜色直方图特征做相似队伍聚类，明显K_mat的队伍分类器是高配版了，Arcface将类间距拉大，增加分类准确性。

(5) Tracker

追踪器记录了所有帧下的球员分配结果，然后再分配到bbox上。作者使用了简单的IoU追踪器，因为它快且足够准确。再分配时，不止参考标签出现次数，还有追踪置信度和帧间距离。分配结果如果远离目标帧，将不被给予权重。

图19：Tracking和结果融合

读后感：这是将WBF应用在追踪结果的融合上。

(6) Ensemble

WBF（Weighted Box Fusion）被用在追踪器再分配阶段中，它集成了多帧多模型的预测结果。将每个模型的球员分配矩阵进行加权平均，然后通过匈牙利算法选择最终分配结果，比常规使用WBF集成bbox和检测置信度更有用。再最终提交中，集成了4个检测器的结果。

图20：WBF集成策略

我们来看下，K_mat实验记录：

图21：第一名的消融实验记录

K_mat最后除了无私地开源了代码[9]，同时，他分享了他的成功拿下第一的原因：

Fortunately, I have the experience of the object detection, depth estimation, pointcloud registration and tracking. I think this is the reason why I could win this competition. I started learning programming and machine learning at kaggle 3 years ago, and most of my knowledge comes from kaggle. Thanks again to the whole of kaggle community!

我看了K_mat的kaggle profile，3年前他的战绩还是1k名左右徘徊，修炼了3年重归kaggle，solo凭49次提交一举拿下第一，令人叹服。

2. 第2名[10]（@tito）

(1) 头盔检测器：使用YoloV5训练头盔检测器，由于小头盔检测效果不好，将图片从1280上采样到1664训练。

(2) 头盔聚类（队伍分类）：使用2阶K-means做头盔聚类。第1个Kmeans从整个视频帧下，所有头盔图片的每个像素集中提取出20个代表性颜色（相当于对所有颜色做聚类20个类）。第2个Kmeans，利用头盔颜色特征向量进行聚类。

(3) 特征抽取（用于距离匹配）：除了2D信息（上/下，左/右，主队/客队，球员朝向），头盔和传感器位置的偏差也被用作球员距离匹配当中。

• 球员朝向：tito构建一个模型预测tracking数据中的球员朝向，然后如果在匹配时，用于惩罚朝向角度差距过大的匹配结果。

图22：球员朝向预测 红色和橙色：对应主队和客队的预测值 浅蓝色和深蓝色：对应主队和客队的真实值

• 头盔与传感器位置偏差：举例，即使是相同坐标，但因为球员站着或蹲着，都会导致头盔位置改变。为此tito构建模型去预测头盔的偏移，他先提前将真实位置和头盔bbox匹配在一起去获取位置偏差，然后训练模型预测距离偏差。

(4) 坐标转换：选手使用OpenCV检测地标线，然后将视频图片和tracking数据的地标线对齐，实现球员坐标系统一。

图23：基于地标线的坐标转换

(5) 球员分配：由于视频中球员数常小于22个，我裁剪掉上下左右的边缘球员（读后感：跟我们一样的做法），在去除冗余球员时，我会选择保留跟其他球员近的球员。

(6) 球员追踪：现在的目标追踪模型是基于图片特征做距离相似度量，然而tito认为此方法也许不适用此比赛，因为同一个队的所有头盔都是同一种头盔，所以tito用了基于SORT的模型（基于KF，而没基于外观特征），然后他也将头盔图片特征作为step-function，用于避免跨球队追踪球员，这里用的图片特征是之前Kmeans队伍聚类时的特征。

读后感：果然猫哥（tito头像是只猫）是老牌选手了，他的方案让人体现了大道至简这四个字。头盔聚类那边的颜色特征提取比我们好，很好的利用了球员朝向数据，同样在头盔与传感器的位置偏差的细节也处理的很好，基于地标线的坐标轴转换很妙，弃DeepSORT改用低配版SORT也非常明智，再一次说明了：最好的不一定是最适合的。

3. 第3名[11]（@fantastic_hirarin）

• (阶段1)使用Yolov5检测头盔。
• (阶段2)使用deepsort追踪头盔。
• (阶段3)使用ICP(迭代最近点)和匈牙利算法将头盔bbox分配给每一帧的跟踪数据。
• (阶段4)基于头盔的颜色，使用kmeans将头盔分为两个类。
• (阶段5)考虑deepsort和头盔颜色信息的结果，使用ICP(迭代最近点)和匈牙利算法分配头盔bbox再次跟踪数据。
• (阶段6)使用匈牙利算法来确定最终的标签。

（1）阶段1： 头盔检测器：基于1280尺寸的图片，训练yolov516和yolov5x，由于担心检测到边缘球员，sideline的头盔不会被用于训练。此外推理时，采用水平翻转做TTA。由于头盔会出现部分可见，为此降低了IOU的置信度阈值。

（2）阶段2：头盔deepsort：基于头盔，用deeposrt追踪器。为了减少假阳样本，删去了连续多帧无法被追踪到的头盔。另外，修改了deepsort代码，让它返回了ReID的抽取特征和置信度。特征会被用在阶段4，置信度会用在阶段3和5。

（3）阶段3：使用ICP和匈牙利算法分配头盔归属：基于检测头盔位置和tracking的球员位置，做ICP尽可能对齐拉近两者位置。然后使用匈牙利算法匹配。同样地，选手也面临假阳样本问题，解决手段是依次删除某个<0.5置信度的样本做ICP（选择匹配距离最小的可能组合），同时也会删除视频画面中边缘多余的检测目标。由于ICP对初始化值有强以来，选手会将上下左右翻转产生多种模式后再进行ICP和匈牙利算法的分配。

图24：使用ICP和匈牙利算法分配头盔归属（阶段3）

（4）阶段4：基于头盔颜色，使用Kmeans聚类：使用了阶段2中deepsort里提取的特征做的聚类。

（5）阶段5：基于头盔bbox和颜色，再做ICP和匈牙利算法匹配：这增加了相同颜色的头盔配分配到相同队伍上的概率。

（6）阶段6：使用匈牙利算法决定最终标签：这里的损失度量标准是在同一个deepsort头盔的标签百分比。显然标签百分比出现最高的，更优先分配。这里没用ICP，是因为此时ICP没法解决头盔检测位置和传感器位置的偏差。

4. 第4名[12]（@Ahmet Erdem）

选手先基于额外图片数据（size=640）训练yolov5头盔检测器，再剔除场外球员H00和V00的视频帧图片（size=1280）fine tune检测器，检测置信权重设为0.03和IOU阈值为0.2。效果好于baseline头盔数据。在Deepsort部分，修改了merge_asof逻辑成贪婪最小距离分配法。加总同一个cluster下的头盔检测置信度作为头盔跟踪置信度，用于选择2D匹配中top22的头盔。Erdem的亮点在于使用resnet34训练球衣号，球衣框是基于头盔bbox外扩得到的，由于模型很小，容易过拟合。选手做了数据增强（cutmix, paste random digit, convert black and white , invert colors），最后球衣号检测器recall小，但precision高，这被加入到2D mapping中，用于重改错误匹配。另外，还针对每个头盔的置信度，由高到低做追踪分配。

对于为什么不用官方提供的baseline头盔预测bbox，原因是他们发现：错误的bbox比缺少bbox 的破坏性更大（因为错误的bbox会破坏整个映射）。所以队员@rytisva88做了两件事：

（1）假设前面一帧的bbox是正确的，那么如果后一帧出现新的bbox，那么你会循环遍历每个新的bbox，然后检查这个新bbox是否修复了前一帧正确bboxes的匹配，如果没有，我们会删掉这个bbox。

（2）检查是否一个bbox从一帧到另外一帧会有消失的情况。做法很类似，就是把你认为是新的bbox，加回来，与未分配的tracking point计算距离，如果距离近，说明这个新的点是“确有其人”。

第5名@Kyle Lee[13]训练了yolov5头盔检测器。在标签分配上，基于加权最小xy距离，使用匈牙利算法。同时，对头50帧做机位投票。deepsort参数偏向于低max age，高iou。其它的基本 没看懂（可能是我太菜了）。 第9名[14]的K-NKSM[]用shape context初始化帧homography matrix还蛮新奇的（虽然没太看懂），第10名nvnn[15]建了一个类似分割网络的回归网络，专门用于匹配视频头盔和tracking数据的球员，非常神奇。第11名[16]的D.Imanishi使用了上一次NFL比赛的光流预测下一帧的bbox，目的是增加头盔数，从而减少bbox的false negatives。

首先，一如既往的感谢队友邵兄和刘兄，从谷歌定位赛再到合作NFL球员分配赛，都很愉快。这次比赛也学到很多，从上面的内容也能看出，这是我今年目前为止打的最有意思，花的最多精力的一次笔试了。前几天我跟队友复盘完，想说如果重来，我们会补充哪些工作，那便是：头盔检测+ICP+匈牙利算法。希望这篇文章能给一些朋友带来乐趣和帮助。

这次也打怪升级成Kaggle Competitions Expert了，希望明年有机会更进一步，摘个银牌。加油！

[1] NFL Health & Safety - Helmet Assignment - Kaggle, 赛题官网: https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/overview

[2] NFL 如何通过机器学习和分析技术实现转型 - AWS, 文章: https:// aws.amazon.com/cn/machi ne-learning/customers/innovators/nfl/

[3] NFL Baseline - Simple Helmet Mapping - tito, Code: https://www. kaggle.com/its7171/nfl- baseline-simple-helmet-mapping/notebook

[4] NFL assign label camera orientation - Adrian Muresan https://www. kaggle.com/louisbunuel/ nfl-assign-label-camera-orientation/notebook

[5] Wojke, N., Bewley, A., & Paulus, D. (2017, September). Simple online and realtime tracking with a deep association metric. In 2017 IEEE international conference on image processing (ICIP) (pp. 3645-3649). IEEE.

[6] DeepSort论文学习 - cdknight_happy, 文章: https:// blog.csdn.net/cdknight_ happy/article/details/79731981

[7] Camera-Tracking Matching with Gradient Descent - Adriano Passos, 代码: https://www. kaggle.com/coldfir3/cam era-tracking-matching-with-gradient-descent

[8] 1st Place Solution - K_mat, 讨论: **** https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/284975

[9] nfl-1stplace-inference - K_mat, 讨论: **** https://www. kaggle.com/kmat2019/nfl -1stplace-inference

[10] 2nd place solution - tito, 讨论: **** https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/285112

[11] [3rd place solution] YOLOv5 + DeepSort + ICP + Hungarian algorithm - fantastic_hirarin, 讨论: **** https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/285076

[12] 4th Place Solution - Ahmet Erdem, 讨论: **** https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/285007

[13] 5th Place Solution - Kyle Lee, 讨论: **** https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/285286

[14] 9th place solution - K-NKSM, 讨论: **** https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/284940

[15] 10th place solution - nvnn, 讨论: **** https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/284945

[16] 11th place solution - D.Imanishi, 讨论: https://www. kaggle.com/c/nfl-health -and-safety-helmet-assignment/discussion/285156

More than 1 year has passed since last update.

Kaggle振り返り NFL Health & Safety - Helmet Assignment

• DeepLearning

　この記事は昨年開催された「NFL Health & Safety - Helmet Assignment」の内容を振り返るために書いています。最後の約１カ月間だけ参加し、私の結果は、61 / 825 (Solo bronze medal, Leaderboard ) でした。個人的には Re-ID の実装を完全に終えることが出来ず、悔しい思いをしたコンペです。手元のテストデータではかなり出来ていたのですが、最終日に Submit するとエラーになりました。あと１日あれば結果はかなり変わっていたと信じています。

Kaggle に コード を公開しています。ご興味のある方は是非。

　毎年開催されているNFL（The National Football League）コンペです。今年のお題は、映像データからアメフト選手のヘルメットを検出し、一意なIDを振り、トラッキングすることでした。私自身は今年が初めての NFL コンペでした。 　各選手は Wearable device を身に着けているため、各選手に選手IDが割り振られており、GPSによってある程度の位置は特定出来ていました。そのため、選手ID、選手の位置（会場を真上から見たときのX, Y）等がコンペ主催者から与えられていましたが、映像データとは紐づいていない状態でした。また、Wearable device から取得されているデータの時間間隔より、映像データの間隔の方が短かった（FPSが大きい）ため、Wearable device のデータをうまく映像データ側に紐づける必要性がありました。 　実際のアメフトを見てみると分かりますが、選手は試合中に色々な体勢をとり、また、選手同士が重なることも多いです。さらに、サイドラインの外側には観客や控え選手がいるため、過検知対策が必要になる等、実際の映像を見てみるとアプローチすべき課題が多かった印象でした。映像は、会場の２方向から撮影されており、撮影方向によっては選手が非常に小さいというのも難しいポイントでした。

　与えられたデータは、画像データ（9,947枚）、映像データ（学習：120ファイル, テスト：6ファイル）、CSVデータ（3種類）です。テストデータの一部が手元にありましたが、Code competition のため、Submit 後に別のテストデータで精度が検証されることになります。 　Metricsは IoU ≧ 0.35 の WeightedAccuracy でした。Weighted の部分で、選手同士が衝突した瞬間を当てれたことと、当てれなかったことに1000倍の差が出るように設計されていました。アメフトにおける人間同士の衝突は非常に危険なため、ここを正確にトラッキングしたいという思いが伺えます。

以下にデータの詳細を記載します。

• 画像データ 過去の色々な試合映像から集められた試合中画像です。このデータはコンペの最終提出物に直接関係なく、使っても使わなくてもOKでした。
• 映像データ 学習用とテスト用の映像データです。１つの映像は数十秒程度から出来ており、ボールが動く（スナップ）10フレーム前から、１つのプレイの終了までに統一されています。映像終了のタイミングは開始ほど厳密には決まっていませんでした。
• ラベル情報（画像データのラベル一覧、学習用映像データのラベル一覧）
• Wearable Device情報（学習・テスト用の選手ID、位置情報等）
• NFL側で作成したヘルメット検出モデルの推論結果です。このCSVが提供されているのは特殊でした。学習・テストの両方のBaselineが手元にあり、これを使っても使わなくても良いという状態でした。

　今回のコンペは大きく２つの問題（検出と紐づけ）で構成されており、それぞれに対して改善作業を実施していきました。実施したことを以下に簡単にまとめます。

• Baseline CSV とのアンサンブル
• ヘルメット識別モデルによる過検知対策
• サイドラインヘルメットの削除

見逃したヘルメットの補完

検出したヘルメットと位置情報の距離によるid割り当て, deep sort アルゴリズムによるre-id, 自作 re-id による選手idの上書き.

　上記に記載した中には、精度向上に繋がったものと、繋がらなかったものの両方があります。また、Public Notebook からそのまま流用したものもあります。

YOLOv5の学習及び Baseline とのアンサンブル

　これは精度向上に繋がりませんでした。YOLOv5 は別のコンペでも利用していたのでコード自体はすぐ完成しました。いくつかのパターンを試しましたが、Baseline そのままで Submit するより毎回精度が悪化するという結果となりました。アンサンブルは、複数のモデルで BBox を出力し、NMS で被っている Bbox を削除するという手法を取りました。 　YOLOv5 を利用すること自体は容易になっており、著者の GitHub や、Kaggle でサンプルコードが多く見つかります。本記事の執筆時点でも YOLOv5 の論文は公開されていませんが、Kaggle 内では多くの利用実績が見られます。

過検知ヘルメットの除外

　実施した２つの方法の内、サイドラインヘルメットの削除によって、Public LB でメダル圏内に入ることが出来ました。１つ目のヘルメット識別モデルの方は改善に寄与しませんでしたが、こちらも含めて簡単に内容を記載します。

ヘルメット識別モデルによる過検知対策 　こちらのモチベーションは、Baseline モデルの Bbox を可視化してみると、選手の肩や足を間違ってヘルメットと認識しているケースが見られたため、ヘルメットかどうかに特化した CNN を作成し、過検出のヘルメットを減らすことにありました。 　まずは、画像データを利用して、ヘルメットを中心として元々の Bbox サイズの数倍を切り出した画像と、ヘルメットの位置からランダムに左、右、下に移動して切り出した画像を作成しました。両者の画像が被る可能性があったので、IoU が大きいものは削除しました。これらの画像に対して、CNN を BCELoss で学習させました。このモデル自体の精度は良かったのですが、これを組み込んで Submit すると精度が悪化するという結果に終わりました。

サイドラインヘルメットの削除 　Baseline の推論結果を映像として見てみると、映像開始直後は出場している選手だけにフォーカスしていたカメラが、選手の移動に伴ってサイドラインの観客や控えの選手を映すようになっていました。この際に、サイドラインの控え選手の多くを過検知していたため、これらの除外を検討することにしました。 　方法としては、プレイの序盤は必ず選手にフォーカスしていることを利用し、最も外側の Bbox 座標を計算し、その座標をトラッキングしていきました。プレイ後半になると、サイドラインのヘルメットが出現し、トラッキングしていた座標が大きく変化します。この変化量が閾値以上だった場合に無視する、という処理を追加することで、ほとんど全てのサイドラインヘルメットを除外出来ました。

　物体検出の特徴として、現在の技術では全てのフレームにおいて人間を正確に捉えることが出来ず、必ず検出したり、見逃したりを繰り返します。ただ、今回は映像のFPSが高かったため、ある選手におけるnフレーム目のヘルメットと、n+1フレーム目のヘルメットは高いIoUになることが期待できました。そのため、下図に示す処理によって見逃したヘルメットを補完しました。 　この例だと、２回目の補完でモデル側が見つけたヘルメットと交差する（高いIoUになる）ことが出来ています。この何回補完を試みるか、と、IoUの閾値がパラメータになっています。今回は、前者は多め（10回）、後者は小さめ（0.2 or 0.3）で実施しました。

　こちらは Public に公開されていた手法をそのまま利用しました。下図のように計算を行う手法です。左が Wearable device で取得されている情報、右が映像データです。距離を計算する際は、 Wearable device 側のデータを数パターン回転させて、最も合計距離が小さい割り当てを採用します。この手法で選手IDを割り当てることが出来ました。 　ただし、ヘルメットの過検知や見逃しがあると大きく割り当て精度が悪くなります。サイドラインヘルメットの除外で精度改善出来たのは、このロジックによる割り当てがうまく挙動したからです。

　上記のロジックによって選手IDの割り当てを行いましたが、割り当て誤りが発生してしまうため、より正確になるよう Deep Sort で再割り当てを行いました。こちらも　Public Notebook の手法をそのまま利用しました。 　Deep Sort は、Re-ID（Person Re-Identification）のためのアルゴリズムです。Deep Sort によって、同一人物と思われる Bbox にGroup ID を振ることが可能です。この Group ID ごとに頻出の選手ID を計算し、上書きを行います。 　アルゴリズムに関する解説は日本語記事 1 もいくつか見つかるので、詳細は割愛し、簡単な説明に留めたいと思います。Deep Sort は、ある人物をトラッキングするために「想定される次フレームでの Bbox の位置」と「同一人物かどうかを判断したい２つの Bbox の特徴量ベクトルの類似度」を基に、同一人物であるかを判定するアルゴリズムです。論文は こちら 。利用したモデルは こちらのGitHub に公開されています。

　上記で使用した Deep Sort の中では同一人物であるかどうか、を学習したモデルを利用しています。しかし、今回の検出対象はヘルメットであったため、Deep Sort では正しく Re-ID 出来ない可能性がありました。再学習という手もありましたが、「見逃したヘルメットの補完」ロジックにおいて、IoUを計算していく過程で Group ID を振ることが出来ると気づき、こちらの実装を優先しました。ロジックを改良して Group ID を割り振り、Group ID ごとに頻出の ID を計算し、上書きを行いました。 　テストデータでの結果を映像として見る限りは、Deep Sort の出力と比べて、かなりの改善が見られました。しかし、冒頭にも記載した通り、Submit するとエラーになってしまい、コンペの時間切れとなってしまいました。Code competition では、Submit 後に別のテストデータでの推論が行われるため、エラーが出るか出ないかは Submit してからでないと分からないという落とし穴があります。

　今回、私としては初めて映像コンペでした。結果的には、Beseline に後処理を追加する感じになり、なんとか銅メダルを獲得することが出来ました。コンペ終了後に公開された 1st place solution は素晴らしいもので、是非みなさん一度読んでみてください。今回も学びの多いコンペでした。参加された皆様、お疲れさまでした。

優勝したKmatさんの Qiita記事 を見つけました！私も2022年はSoloホームランを狙いたいと思います。

参考：Deep Sort の 日本語解説記事 ↩

Register as a new user and use Qiita more conveniently

• You get articles that match your needs
• You can efficiently read back useful information
• You can use dark theme

NFL will allow players to wear Guardian Cap helmets in games

A n NFL rule change will allow players to wear Guardian Cap safety helmets in regular-season games starting next season, the padded headgear offering better protection from brain injuries.

The move was mentioned in a league webinar on NFL safety and medical issues two weeks ago but was not revealed in a major way until Friday.

"We now have two years of data showing significant concussion reductions among players who wear Guardian Caps during practice so players will be permitted to wear the cap during games this upcoming season," Jeff Miller, the NFL executive vice president for player health and safety, said Friday in a statement.

"Additionally, there are new helmets this year that provide as much –- if not more –- protection than a different helmet model paired with a Guardian Cap. These developments represent substantial progress in our efforts to make the game safer for players."

The Guardian Caps were required for players in certain positions for 2022 and 2023 training camps and have resulted in almost a 50% reduction in concussions for those players who wore them.

Dawn Aponte, the NFL's chief football administrative officer, said in an April 9 webinar that the helmets would be permitted in games.

"There is the option for a player to wear it in a game if he so chooses," she said. "There were a number of clubs that had already required all of their players to wear those."

NFL chief medical officer Allen Sills reacted to the news, saying, "So we might actually see a Guardian Cap on a player in a game this year. That's possible. Great. Big change coming up."

The league had previously not allowed players to use the Guardian Caps during games.

League information gathered over the past few years has found that 10% of the force for any hit to the helmet would be absorbed by the padded cap if used and if two players wearing the cap were involved in the hit, 20% of the impact would be absorbed in the padding.

The NFL and NFL Players Association earlier this month announced five new helmets were approved for the 2024 season after testing better for protection than any other helmet, providing a record 12 helmet models for the upcoming season -- including more position-specific helmets designed to mitigate impacts more likely by those playing specific positions.