diff --git a/Chinese-multiperson-voice-recognition-transfer-learning/LICENSE b/Chinese-multiperson-voice-recognition-transfer-learning/LICENSE index 3877ae0a..f288702d 100644 --- a/Chinese-multiperson-voice-recognition-transfer-learning/LICENSE +++ b/Chinese-multiperson-voice-recognition-transfer-learning/LICENSE @@ -1,674 +1,674 @@ - GNU GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - - Copyright (C) 2007 Free Software Foundation, Inc. - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The GNU General Public License is a free, copyleft license for -software and other kinds of works. - - The licenses for most software and other practical works are designed -to take away your freedom to share and change the works. By contrast, -the GNU General Public License is intended to guarantee your freedom to -share and change all versions of a program--to make sure it remains free -software for all its users. We, the Free Software Foundation, use the -GNU General Public License for most of our software; it applies also to -any other work released this way by its authors. You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -them if you wish), that you receive source code or can get it if you -want it, that you can change the software or use pieces of it in new -free programs, and that you know you can do these things. - - To protect your rights, we need to prevent others from denying you -these rights or asking you to surrender the rights. Therefore, you have -certain responsibilities if you distribute copies of the software, or if -you modify it: responsibilities to respect the freedom of others. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must pass on to the recipients the same -freedoms that you received. You must make sure that they, too, receive -or can get the source code. And you must show them these terms so they -know their rights. - - Developers that use the GNU GPL protect your rights with two steps: -(1) assert copyright on the software, and (2) offer you this License -giving you legal permission to copy, distribute and/or modify it. - - For the developers' and authors' protection, the GPL clearly explains -that there is no warranty for this free software. For both users' and -authors' sake, the GPL requires that modified versions be marked as -changed, so that their problems will not be attributed erroneously to -authors of previous versions. - - Some devices are designed to deny users access to install or run -modified versions of the software inside them, although the manufacturer -can do so. This is fundamentally incompatible with the aim of -protecting users' freedom to change the software. The systematic -pattern of such abuse occurs in the area of products for individuals to -use, which is precisely where it is most unacceptable. Therefore, we -have designed this version of the GPL to prohibit the practice for those -products. If such problems arise substantially in other domains, we -stand ready to extend this provision to those domains in future versions -of the GPL, as needed to protect the freedom of users. - - Finally, every program is threatened constantly by software patents. -States should not allow patents to restrict development and use of -software on general-purpose computers, but in those that do, we wish to -avoid the special danger that patents applied to a free program could -make it effectively proprietary. To prevent this, the GPL assures that -patents cannot be used to render the program non-free. - - The precise terms and conditions for copying, distribution and -modification follow. - - TERMS AND CONDITIONS - - 0. Definitions. - - "This License" refers to version 3 of the GNU General Public License. - - "Copyright" also means copyright-like laws that apply to other kinds of -works, such as semiconductor masks. - - "The Program" refers to any copyrightable work licensed under this -License. Each licensee is addressed as "you". "Licensees" and -"recipients" may be individuals or organizations. - - To "modify" a work means to copy from or adapt all or part of the work -in a fashion requiring copyright permission, other than the making of an -exact copy. The resulting work is called a "modified version" of the -earlier work or a work "based on" the earlier work. - - A "covered work" means either the unmodified Program or a work based -on the Program. - - To "propagate" a work means to do anything with it that, without -permission, would make you directly or secondarily liable for -infringement under applicable copyright law, except executing it on a -computer or modifying a private copy. Propagation includes copying, -distribution (with or without modification), making available to the -public, and in some countries other activities as well. - - To "convey" a work means any kind of propagation that enables other -parties to make or receive copies. Mere interaction with a user through -a computer network, with no transfer of a copy, is not conveying. - - An interactive user interface displays "Appropriate Legal Notices" -to the extent that it includes a convenient and prominently visible -feature that (1) displays an appropriate copyright notice, and (2) -tells the user that there is no warranty for the work (except to the -extent that warranties are provided), that licensees may convey the -work under this License, and how to view a copy of this License. If -the interface presents a list of user commands or options, such as a -menu, a prominent item in the list meets this criterion. - - 1. Source Code. - - The "source code" for a work means the preferred form of the work -for making modifications to it. "Object code" means any non-source -form of a work. - - A "Standard Interface" means an interface that either is an official -standard defined by a recognized standards body, or, in the case of -interfaces specified for a particular programming language, one that -is widely used among developers working in that language. - - The "System Libraries" of an executable work include anything, other -than the work as a whole, that (a) is included in the normal form of -packaging a Major Component, but which is not part of that Major -Component, and (b) serves only to enable use of the work with that -Major Component, or to implement a Standard Interface for which an -implementation is available to the public in source code form. A -"Major Component", in this context, means a major essential component -(kernel, window system, and so on) of the specific operating system -(if any) on which the executable work runs, or a compiler used to -produce the work, or an object code interpreter used to run it. - - The "Corresponding Source" for a work in object code form means all -the source code needed to generate, install, and (for an executable -work) run the object code and to modify the work, including scripts to -control those activities. However, it does not include the work's -System Libraries, or general-purpose tools or generally available free -programs which are used unmodified in performing those activities but -which are not part of the work. For example, Corresponding Source -includes interface definition files associated with source files for -the work, and the source code for shared libraries and dynamically -linked subprograms that the work is specifically designed to require, -such as by intimate data communication or control flow between those -subprograms and other parts of the work. - - The Corresponding Source need not include anything that users -can regenerate automatically from other parts of the Corresponding -Source. - - The Corresponding Source for a work in source code form is that -same work. - - 2. Basic Permissions. - - All rights granted under this License are granted for the term of -copyright on the Program, and are irrevocable provided the stated -conditions are met. This License explicitly affirms your unlimited -permission to run the unmodified Program. The output from running a -covered work is covered by this License only if the output, given its -content, constitutes a covered work. This License acknowledges your -rights of fair use or other equivalent, as provided by copyright law. - - You may make, run and propagate covered works that you do not -convey, without conditions so long as your license otherwise remains -in force. You may convey covered works to others for the sole purpose -of having them make modifications exclusively for you, or provide you -with facilities for running those works, provided that you comply with -the terms of this License in conveying all material for which you do -not control copyright. Those thus making or running the covered works -for you must do so exclusively on your behalf, under your direction -and control, on terms that prohibit them from making any copies of -your copyrighted material outside their relationship with you. - - Conveying under any other circumstances is permitted solely under -the conditions stated below. Sublicensing is not allowed; section 10 -makes it unnecessary. - - 3. Protecting Users' Legal Rights From Anti-Circumvention Law. - - No covered work shall be deemed part of an effective technological -measure under any applicable law fulfilling obligations under article -11 of the WIPO copyright treaty adopted on 20 December 1996, or -similar laws prohibiting or restricting circumvention of such -measures. - - When you convey a covered work, you waive any legal power to forbid -circumvention of technological measures to the extent such circumvention -is effected by exercising rights under this License with respect to -the covered work, and you disclaim any intention to limit operation or -modification of the work as a means of enforcing, against the work's -users, your or third parties' legal rights to forbid circumvention of -technological measures. - - 4. Conveying Verbatim Copies. - - You may convey verbatim copies of the Program's source code as you -receive it, in any medium, provided that you conspicuously and -appropriately publish on each copy an appropriate copyright notice; -keep intact all notices stating that this License and any -non-permissive terms added in accord with section 7 apply to the code; -keep intact all notices of the absence of any warranty; and give all -recipients a copy of this License along with the Program. - - You may charge any price or no price for each copy that you convey, -and you may offer support or warranty protection for a fee. - - 5. Conveying Modified Source Versions. - - You may convey a work based on the Program, or the modifications to -produce it from the Program, in the form of source code under the -terms of section 4, provided that you also meet all of these conditions: - - a) The work must carry prominent notices stating that you modified - it, and giving a relevant date. - - b) The work must carry prominent notices stating that it is - released under this License and any conditions added under section - 7. This requirement modifies the requirement in section 4 to - "keep intact all notices". - - c) You must license the entire work, as a whole, under this - License to anyone who comes into possession of a copy. This - License will therefore apply, along with any applicable section 7 - additional terms, to the whole of the work, and all its parts, - regardless of how they are packaged. This License gives no - permission to license the work in any other way, but it does not - invalidate such permission if you have separately received it. - - d) If the work has interactive user interfaces, each must display - Appropriate Legal Notices; however, if the Program has interactive - interfaces that do not display Appropriate Legal Notices, your - work need not make them do so. - - A compilation of a covered work with other separate and independent -works, which are not by their nature extensions of the covered work, -and which are not combined with it such as to form a larger program, -in or on a volume of a storage or distribution medium, is called an -"aggregate" if the compilation and its resulting copyright are not -used to limit the access or legal rights of the compilation's users -beyond what the individual works permit. Inclusion of a covered work -in an aggregate does not cause this License to apply to the other -parts of the aggregate. - - 6. Conveying Non-Source Forms. - - You may convey a covered work in object code form under the terms -of sections 4 and 5, provided that you also convey the -machine-readable Corresponding Source under the terms of this License, -in one of these ways: - - a) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by the - Corresponding Source fixed on a durable physical medium - customarily used for software interchange. - - b) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by a - written offer, valid for at least three years and valid for as - long as you offer spare parts or customer support for that product - model, to give anyone who possesses the object code either (1) a - copy of the Corresponding Source for all the software in the - product that is covered by this License, on a durable physical - medium customarily used for software interchange, for a price no - more than your reasonable cost of physically performing this - conveying of source, or (2) access to copy the - Corresponding Source from a network server at no charge. - - c) Convey individual copies of the object code with a copy of the - written offer to provide the Corresponding Source. This - alternative is allowed only occasionally and noncommercially, and - only if you received the object code with such an offer, in accord - with subsection 6b. - - d) Convey the object code by offering access from a designated - place (gratis or for a charge), and offer equivalent access to the - Corresponding Source in the same way through the same place at no - further charge. You need not require recipients to copy the - Corresponding Source along with the object code. If the place to - copy the object code is a network server, the Corresponding Source - may be on a different server (operated by you or a third party) - that supports equivalent copying facilities, provided you maintain - clear directions next to the object code saying where to find the - Corresponding Source. Regardless of what server hosts the - Corresponding Source, you remain obligated to ensure that it is - available for as long as needed to satisfy these requirements. - - e) Convey the object code using peer-to-peer transmission, provided - you inform other peers where the object code and Corresponding - Source of the work are being offered to the general public at no - charge under subsection 6d. - - A separable portion of the object code, whose source code is excluded -from the Corresponding Source as a System Library, need not be -included in conveying the object code work. - - A "User Product" is either (1) a "consumer product", which means any -tangible personal property which is normally used for personal, family, -or household purposes, or (2) anything designed or sold for incorporation -into a dwelling. In determining whether a product is a consumer product, -doubtful cases shall be resolved in favor of coverage. For a particular -product received by a particular user, "normally used" refers to a -typical or common use of that class of product, regardless of the status -of the particular user or of the way in which the particular user -actually uses, or expects or is expected to use, the product. A product -is a consumer product regardless of whether the product has substantial -commercial, industrial or non-consumer uses, unless such uses represent -the only significant mode of use of the product. - - "Installation Information" for a User Product means any methods, -procedures, authorization keys, or other information required to install -and execute modified versions of a covered work in that User Product from -a modified version of its Corresponding Source. The information must -suffice to ensure that the continued functioning of the modified object -code is in no case prevented or interfered with solely because -modification has been made. - - If you convey an object code work under this section in, or with, or -specifically for use in, a User Product, and the conveying occurs as -part of a transaction in which the right of possession and use of the -User Product is transferred to the recipient in perpetuity or for a -fixed term (regardless of how the transaction is characterized), the -Corresponding Source conveyed under this section must be accompanied -by the Installation Information. But this requirement does not apply -if neither you nor any third party retains the ability to install -modified object code on the User Product (for example, the work has -been installed in ROM). - - The requirement to provide Installation Information does not include a -requirement to continue to provide support service, warranty, or updates -for a work that has been modified or installed by the recipient, or for -the User Product in which it has been modified or installed. Access to a -network may be denied when the modification itself materially and -adversely affects the operation of the network or violates the rules and -protocols for communication across the network. - - Corresponding Source conveyed, and Installation Information provided, -in accord with this section must be in a format that is publicly -documented (and with an implementation available to the public in -source code form), and must require no special password or key for -unpacking, reading or copying. - - 7. Additional Terms. - - "Additional permissions" are terms that supplement the terms of this -License by making exceptions from one or more of its conditions. -Additional permissions that are applicable to the entire Program shall -be treated as though they were included in this License, to the extent -that they are valid under applicable law. If additional permissions -apply only to part of the Program, that part may be used separately -under those permissions, but the entire Program remains governed by -this License without regard to the additional permissions. - - When you convey a copy of a covered work, you may at your option -remove any additional permissions from that copy, or from any part of -it. (Additional permissions may be written to require their own -removal in certain cases when you modify the work.) You may place -additional permissions on material, added by you to a covered work, -for which you have or can give appropriate copyright permission. - - Notwithstanding any other provision of this License, for material you -add to a covered work, you may (if authorized by the copyright holders of -that material) supplement the terms of this License with terms: - - a) Disclaiming warranty or limiting liability differently from the - terms of sections 15 and 16 of this License; or - - b) Requiring preservation of specified reasonable legal notices or - author attributions in that material or in the Appropriate Legal - Notices displayed by works containing it; or - - c) Prohibiting misrepresentation of the origin of that material, or - requiring that modified versions of such material be marked in - reasonable ways as different from the original version; or - - d) Limiting the use for publicity purposes of names of licensors or - authors of the material; or - - e) Declining to grant rights under trademark law for use of some - trade names, trademarks, or service marks; or - - f) Requiring indemnification of licensors and authors of that - material by anyone who conveys the material (or modified versions of - it) with contractual assumptions of liability to the recipient, for - any liability that these contractual assumptions directly impose on - those licensors and authors. - - All other non-permissive additional terms are considered "further -restrictions" within the meaning of section 10. If the Program as you -received it, or any part of it, contains a notice stating that it is -governed by this License along with a term that is a further -restriction, you may remove that term. If a license document contains -a further restriction but permits relicensing or conveying under this -License, you may add to a covered work material governed by the terms -of that license document, provided that the further restriction does -not survive such relicensing or conveying. - - If you add terms to a covered work in accord with this section, you -must place, in the relevant source files, a statement of the -additional terms that apply to those files, or a notice indicating -where to find the applicable terms. - - Additional terms, permissive or non-permissive, may be stated in the -form of a separately written license, or stated as exceptions; -the above requirements apply either way. - - 8. Termination. - - You may not propagate or modify a covered work except as expressly -provided under this License. Any attempt otherwise to propagate or -modify it is void, and will automatically terminate your rights under -this License (including any patent licenses granted under the third -paragraph of section 11). - - However, if you cease all violation of this License, then your -license from a particular copyright holder is reinstated (a) -provisionally, unless and until the copyright holder explicitly and -finally terminates your license, and (b) permanently, if the copyright -holder fails to notify you of the violation by some reasonable means -prior to 60 days after the cessation. - - Moreover, your license from a particular copyright holder is -reinstated permanently if the copyright holder notifies you of the -violation by some reasonable means, this is the first time you have -received notice of violation of this License (for any work) from that -copyright holder, and you cure the violation prior to 30 days after -your receipt of the notice. - - Termination of your rights under this section does not terminate the -licenses of parties who have received copies or rights from you under -this License. If your rights have been terminated and not permanently -reinstated, you do not qualify to receive new licenses for the same -material under section 10. - - 9. Acceptance Not Required for Having Copies. - - You are not required to accept this License in order to receive or -run a copy of the Program. Ancillary propagation of a covered work -occurring solely as a consequence of using peer-to-peer transmission -to receive a copy likewise does not require acceptance. However, -nothing other than this License grants you permission to propagate or -modify any covered work. These actions infringe copyright if you do -not accept this License. Therefore, by modifying or propagating a -covered work, you indicate your acceptance of this License to do so. - - 10. Automatic Licensing of Downstream Recipients. - - Each time you convey a covered work, the recipient automatically -receives a license from the original licensors, to run, modify and -propagate that work, subject to this License. You are not responsible -for enforcing compliance by third parties with this License. - - An "entity transaction" is a transaction transferring control of an -organization, or substantially all assets of one, or subdividing an -organization, or merging organizations. If propagation of a covered -work results from an entity transaction, each party to that -transaction who receives a copy of the work also receives whatever -licenses to the work the party's predecessor in interest had or could -give under the previous paragraph, plus a right to possession of the -Corresponding Source of the work from the predecessor in interest, if -the predecessor has it or can get it with reasonable efforts. - - You may not impose any further restrictions on the exercise of the -rights granted or affirmed under this License. For example, you may -not impose a license fee, royalty, or other charge for exercise of -rights granted under this License, and you may not initiate litigation -(including a cross-claim or counterclaim in a lawsuit) alleging that -any patent claim is infringed by making, using, selling, offering for -sale, or importing the Program or any portion of it. - - 11. Patents. - - A "contributor" is a copyright holder who authorizes use under this -License of the Program or a work on which the Program is based. The -work thus licensed is called the contributor's "contributor version". - - A contributor's "essential patent claims" are all patent claims -owned or controlled by the contributor, whether already acquired or -hereafter acquired, that would be infringed by some manner, permitted -by this License, of making, using, or selling its contributor version, -but do not include claims that would be infringed only as a -consequence of further modification of the contributor version. For -purposes of this definition, "control" includes the right to grant -patent sublicenses in a manner consistent with the requirements of -this License. - - Each contributor grants you a non-exclusive, worldwide, royalty-free -patent license under the contributor's essential patent claims, to -make, use, sell, offer for sale, import and otherwise run, modify and -propagate the contents of its contributor version. - - In the following three paragraphs, a "patent license" is any express -agreement or commitment, however denominated, not to enforce a patent -(such as an express permission to practice a patent or covenant not to -sue for patent infringement). To "grant" such a patent license to a -party means to make such an agreement or commitment not to enforce a -patent against the party. - - If you convey a covered work, knowingly relying on a patent license, -and the Corresponding Source of the work is not available for anyone -to copy, free of charge and under the terms of this License, through a -publicly available network server or other readily accessible means, -then you must either (1) cause the Corresponding Source to be so -available, or (2) arrange to deprive yourself of the benefit of the -patent license for this particular work, or (3) arrange, in a manner -consistent with the requirements of this License, to extend the patent -license to downstream recipients. "Knowingly relying" means you have -actual knowledge that, but for the patent license, your conveying the -covered work in a country, or your recipient's use of the covered work -in a country, would infringe one or more identifiable patents in that -country that you have reason to believe are valid. - - If, pursuant to or in connection with a single transaction or -arrangement, you convey, or propagate by procuring conveyance of, a -covered work, and grant a patent license to some of the parties -receiving the covered work authorizing them to use, propagate, modify -or convey a specific copy of the covered work, then the patent license -you grant is automatically extended to all recipients of the covered -work and works based on it. - - A patent license is "discriminatory" if it does not include within -the scope of its coverage, prohibits the exercise of, or is -conditioned on the non-exercise of one or more of the rights that are -specifically granted under this License. You may not convey a covered -work if you are a party to an arrangement with a third party that is -in the business of distributing software, under which you make payment -to the third party based on the extent of your activity of conveying -the work, and under which the third party grants, to any of the -parties who would receive the covered work from you, a discriminatory -patent license (a) in connection with copies of the covered work -conveyed by you (or copies made from those copies), or (b) primarily -for and in connection with specific products or compilations that -contain the covered work, unless you entered into that arrangement, -or that patent license was granted, prior to 28 March 2007. - - Nothing in this License shall be construed as excluding or limiting -any implied license or other defenses to infringement that may -otherwise be available to you under applicable patent law. - - 12. No Surrender of Others' Freedom. - - If conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot convey a -covered work so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you may -not convey it at all. For example, if you agree to terms that obligate you -to collect a royalty for further conveying from those to whom you convey -the Program, the only way you could satisfy both those terms and this -License would be to refrain entirely from conveying the Program. - - 13. Use with the GNU Affero General Public License. - - Notwithstanding any other provision of this License, you have -permission to link or combine any covered work with a work licensed -under version 3 of the GNU Affero General Public License into a single -combined work, and to convey the resulting work. The terms of this -License will continue to apply to the part which is the covered work, -but the special requirements of the GNU Affero General Public License, -section 13, concerning interaction through a network will apply to the -combination as such. - - 14. Revised Versions of this License. - - The Free Software Foundation may publish revised and/or new versions of -the GNU General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - - Each version is given a distinguishing version number. If the -Program specifies that a certain numbered version of the GNU General -Public License "or any later version" applies to it, you have the -option of following the terms and conditions either of that numbered -version or of any later version published by the Free Software -Foundation. If the Program does not specify a version number of the -GNU General Public License, you may choose any version ever published -by the Free Software Foundation. - - If the Program specifies that a proxy can decide which future -versions of the GNU General Public License can be used, that proxy's -public statement of acceptance of a version permanently authorizes you -to choose that version for the Program. - - Later license versions may give you additional or different -permissions. However, no additional obligations are imposed on any -author or copyright holder as a result of your choosing to follow a -later version. - - 15. Disclaimer of Warranty. - - THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY -APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT -HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY -OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, -THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR -PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM -IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF -ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - - 16. Limitation of Liability. - - IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS -THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY -GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE -USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF -DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD -PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), -EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF -SUCH DAMAGES. - - 17. Interpretation of Sections 15 and 16. - - If the disclaimer of warranty and limitation of liability provided -above cannot be given local legal effect according to their terms, -reviewing courts shall apply local law that most closely approximates -an absolute waiver of all civil liability in connection with the -Program, unless a warranty or assumption of liability accompanies a -copy of the Program in return for a fee. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -state the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . - -Also add information on how to contact you by electronic and paper mail. - - If the program does terminal interaction, make it output a short -notice like this when it starts in an interactive mode: - - Copyright (C) - This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, your program's commands -might be different; for a GUI interface, you would use an "about box". - - You should also get your employer (if you work as a programmer) or school, -if any, to sign a "copyright disclaimer" for the program, if necessary. -For more information on this, and how to apply and follow the GNU GPL, see -. - - The GNU General Public License does not permit incorporating your program -into proprietary programs. If your program is a subroutine library, you -may consider it more useful to permit linking proprietary applications with -the library. If this is what you want to do, use the GNU Lesser General -Public License instead of this License. But first, please read -. + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/Chinese-multiperson-voice-recognition-transfer-learning/README.md b/Chinese-multiperson-voice-recognition-transfer-learning/README.md index 6719e5c6..d2e5fa00 100644 --- a/Chinese-multiperson-voice-recognition-transfer-learning/README.md +++ b/Chinese-multiperson-voice-recognition-transfer-learning/README.md @@ -1,68 +1,68 @@ -# Chinese-multiperson-voice-recognition-using-transfer-learning -This is an example of applying transfer learning to the Chinese multi-person voice recognition application. Transfer learning is an AI technique used to enhance the training accuracy of use cases when the dataset is small or the training accuracy is low given the high noise of the original dataset. Multi-person voice recognition is known to contain high noise in the dataset. Chinese voice voice recognition has gained much progress recently thanks to the effort by the big name company such as Google. However many issues remain unsolved. Multi-person Chinese voice recognition is one of them. This example provieds not only multi-person Chinese voice sample dataset, but applied a transfer learning technique to the CNN trained model of the Chinese voice samples dataset. Satisfactory results can be achieved through transfer learning after an initial CNN training. -This example provides a feasibility evidence of the transfer learning techniques, and it is our wish to convert the transfer learning technique to a Kubeflow asset through this illustration case. A transfer learning pipeline will be constructed to make kubeflow user easy to adapt to their model for training accuracy enhancement. Eventually, other users can benefit from such convenient features of the kubeflow resources. - -usage briefing: -1.Process audio files and convert them into spectrograms. -2.Establish experimental data, divide them into 3 categories, and set them into CNN network training. -3.Perform two training sessions to improve accuracy. -4.Compare training methods. - -Tools used: -1. TensorFlow -2. Anaconda -3. Python3.7 - -1. preprocess(spectrograms production) - -![image](https://user-images.githubusercontent.com/58965086/122675714-48a34700-d20d-11eb-81d7-865209ac8367.png) - -2. import spectrogram files. -![image](https://user-images.githubusercontent.com/58965086/122675748-712b4100-d20d-11eb-96cd-1523b9329020.png) - -![image](https://user-images.githubusercontent.com/58965086/122675762-8011f380-d20d-11eb-90db-f7b8942571d5.png) - -3. build training dataset: -divide the dataset into training, validation, and testing sets. - -![image](https://user-images.githubusercontent.com/58965086/122675818-b8b1cd00-d20d-11eb-836a-08fa3e870823.png) - -4. build CNN taining: - -![image](https://user-images.githubusercontent.com/58965086/122675838-d54e0500-d20d-11eb-8076-8dc78600a779.png) - -5. first training -![image](https://user-images.githubusercontent.com/58965086/122675851-e565e480-d20d-11eb-8ad4-4dada12f70a0.png) -![image](https://user-images.githubusercontent.com/58965086/122675854-ea2a9880-d20d-11eb-82f8-4ad9fc506386.png) - -6. first training result: -![image](https://user-images.githubusercontent.com/58965086/122675877-03cbe000-d20e-11eb-8f64-1c4ad9cec5a8.png) - -7. visualize the result -![image](https://user-images.githubusercontent.com/58965086/122675890-1b0acd80-d20e-11eb-84da-1793cac58fe2.png) - -![image](https://user-images.githubusercontent.com/58965086/122675896-2100ae80-d20e-11eb-8901-240b7d9b3566.png) - -8. import VGG16 model -![image](https://user-images.githubusercontent.com/58965086/122675911-37a70580-d20e-11eb-95ab-a4a652fa79ab.png) -![image](https://user-images.githubusercontent.com/58965086/122675916-3e357d00-d20e-11eb-99ab-a5d22facba9c.png) - -9. use conv_base model to extract features and labels -![image](https://user-images.githubusercontent.com/58965086/122675969-7937b080-d20e-11eb-885d-c5f202b3457a.png) - -10. training -![image](https://user-images.githubusercontent.com/58965086/122676010-92d8f800-d20e-11eb-8826-5b599bc78b4b.png) - -11. visualize the results -![image](https://user-images.githubusercontent.com/58965086/122676032-a71cf500-d20e-11eb-82fc-a2a468340a53.png) - -12. build confusion matrix -![image](https://user-images.githubusercontent.com/58965086/122676055-c1ef6980-d20e-11eb-81af-c394696124c7.png) - -13. visualiz the confusion matrix -![image](https://user-images.githubusercontent.com/58965086/122676069-d7fd2a00-d20e-11eb-8dcb-064f8406e9a4.png) - -14. sample Chinese multiperson voice spectrogram files are added -15. sample VGG16 transfer learning code: vgg16.ipynb is added to the repository - - +# Chinese-multiperson-voice-recognition-using-transfer-learning +This is an example of applying transfer learning to the Chinese multi-person voice recognition application. Transfer learning is an AI technique used to enhance the training accuracy of use cases when the dataset is small or the training accuracy is low given the high noise of the original dataset. Multi-person voice recognition is known to contain high noise in the dataset. Chinese voice voice recognition has gained much progress recently thanks to the effort by the big name company such as Google. However many issues remain unsolved. Multi-person Chinese voice recognition is one of them. This example provieds not only multi-person Chinese voice sample dataset, but applied a transfer learning technique to the CNN trained model of the Chinese voice samples dataset. Satisfactory results can be achieved through transfer learning after an initial CNN training. +This example provides a feasibility evidence of the transfer learning techniques, and it is our wish to convert the transfer learning technique to a Kubeflow asset through this illustration case. A transfer learning pipeline will be constructed to make kubeflow user easy to adapt to their model for training accuracy enhancement. Eventually, other users can benefit from such convenient features of the kubeflow resources. + +usage briefing: +1.Process audio files and convert them into spectrograms. +2.Establish experimental data, divide them into 3 categories, and set them into CNN network training. +3.Perform two training sessions to improve accuracy. +4.Compare training methods. + +Tools used: +1. TensorFlow +2. Anaconda +3. Python3.7 + +1. preprocess(spectrograms production) + +![image](https://user-images.githubusercontent.com/58965086/122675714-48a34700-d20d-11eb-81d7-865209ac8367.png) + +2. import spectrogram files. +![image](https://user-images.githubusercontent.com/58965086/122675748-712b4100-d20d-11eb-96cd-1523b9329020.png) + +![image](https://user-images.githubusercontent.com/58965086/122675762-8011f380-d20d-11eb-90db-f7b8942571d5.png) + +3. build training dataset: +divide the dataset into training, validation, and testing sets. + +![image](https://user-images.githubusercontent.com/58965086/122675818-b8b1cd00-d20d-11eb-836a-08fa3e870823.png) + +4. build CNN taining: + +![image](https://user-images.githubusercontent.com/58965086/122675838-d54e0500-d20d-11eb-8076-8dc78600a779.png) + +5. first training +![image](https://user-images.githubusercontent.com/58965086/122675851-e565e480-d20d-11eb-8ad4-4dada12f70a0.png) +![image](https://user-images.githubusercontent.com/58965086/122675854-ea2a9880-d20d-11eb-82f8-4ad9fc506386.png) + +6. first training result: +![image](https://user-images.githubusercontent.com/58965086/122675877-03cbe000-d20e-11eb-8f64-1c4ad9cec5a8.png) + +7. visualize the result +![image](https://user-images.githubusercontent.com/58965086/122675890-1b0acd80-d20e-11eb-84da-1793cac58fe2.png) + +![image](https://user-images.githubusercontent.com/58965086/122675896-2100ae80-d20e-11eb-8901-240b7d9b3566.png) + +8. import VGG16 model +![image](https://user-images.githubusercontent.com/58965086/122675911-37a70580-d20e-11eb-95ab-a4a652fa79ab.png) +![image](https://user-images.githubusercontent.com/58965086/122675916-3e357d00-d20e-11eb-99ab-a5d22facba9c.png) + +9. use conv_base model to extract features and labels +![image](https://user-images.githubusercontent.com/58965086/122675969-7937b080-d20e-11eb-885d-c5f202b3457a.png) + +10. training +![image](https://user-images.githubusercontent.com/58965086/122676010-92d8f800-d20e-11eb-8826-5b599bc78b4b.png) + +11. visualize the results +![image](https://user-images.githubusercontent.com/58965086/122676032-a71cf500-d20e-11eb-82fc-a2a468340a53.png) + +12. build confusion matrix +![image](https://user-images.githubusercontent.com/58965086/122676055-c1ef6980-d20e-11eb-81af-c394696124c7.png) + +13. visualiz the confusion matrix +![image](https://user-images.githubusercontent.com/58965086/122676069-d7fd2a00-d20e-11eb-8dcb-064f8406e9a4.png) + +14. sample Chinese multiperson voice spectrogram files are added +15. sample VGG16 transfer learning code: vgg16.ipynb is added to the repository + + diff --git a/Chinese-multiperson-voice-recognition-transfer-learning/vgg16.ipynb b/Chinese-multiperson-voice-recognition-transfer-learning/vgg16.ipynb index 86652b0a..b87ba944 100644 --- a/Chinese-multiperson-voice-recognition-transfer-learning/vgg16.ipynb +++ b/Chinese-multiperson-voice-recognition-transfer-learning/vgg16.ipynb @@ -1,1008 +1,1008 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "#匯入訓練、驗證、測試圖像\n", - "import os,shutil\n", - "\n", - "base_dir = 'Rec'\n", - "if not os.path.isdir(base_dir):\n", - " os.mkdir(base_dir)\n", - " \n", - "train_dir = os.path.join(base_dir, 'train')\n", - "os.mkdir(train_dir)\n", - "validation_dir = os.path.join(base_dir, 'validation')\n", - "os.mkdir(validation_dir)\n", - "test_dir = os.path.join(base_dir, 'test')\n", - "os.mkdir(test_dir)" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "#建立訓練集資料夾\n", - "train_b_dir = os.path.join(train_dir, 'be')\n", - "os.mkdir(train_b_dir)\n", - "\n", - "train_h_dir = os.path.join(train_dir, 'ho')\n", - "os.mkdir(train_h_dir)\n", - "\n", - "\n", - "train_y_dir = os.path.join(train_dir, 'yun')\n", - "os.mkdir(train_y_dir)" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], - "source": [ - "#建立驗證集資料夾\n", - "validation_b_dir = os.path.join(validation_dir, 'be')\n", - "os.mkdir(validation_b_dir)\n", - "\n", - "validation_h_dir = os.path.join(validation_dir, 'ho')\n", - "os.mkdir(validation_h_dir)\n", - "\n", - "validation_y_dir = os.path.join(validation_dir, 'yun')\n", - "os.mkdir(validation_y_dir)\n" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "#建立測試集資料夾\n", - "test_b_dir = os.path.join(test_dir, 'be')\n", - "os.mkdir(test_b_dir)\n", - "\n", - "test_h_dir = os.path.join(test_dir, 'ho')\n", - "os.mkdir(test_h_dir)\n", - "\n", - "test_y_dir = os.path.join(test_dir, 'yun')\n", - "os.mkdir(test_y_dir)\n" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [], - "source": [ - "#擷取be資料集的圖片範圍\n", - "original_0dataset_dir = 'be'\n", - "fnames = ['be.{}.png'.format(i) for i in range(20)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(train_b_dir, fname)\n", - " shutil.copyfile(src, dst)\n", - "\n", - "fnames = ['be.{}.png'.format(i) for i in range(45, 54)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(validation_b_dir, fname)\n", - " shutil.copyfile(src, dst)\n", - "\n", - "fnames = ['be.{}.png'.format(i) for i in range(54, 59)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(test_b_dir, fname)\n", - " shutil.copyfile(src, dst)" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [], - "source": [ - "#擷取ho資料集的圖片範圍\n", - "original_0dataset_dir = 'ho'\n", - "fnames = ['ho.{}.png'.format(i) for i in range(20)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(train_h_dir, fname)\n", - " shutil.copyfile(src, dst)\n", - "\n", - "fnames = ['ho.{}.png'.format(i) for i in range(45, 54)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(validation_h_dir, fname)\n", - " shutil.copyfile(src, dst)\n", - "\n", - "fnames = ['ho.{}.png'.format(i) for i in range(54, 59)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(test_h_dir, fname)\n", - " shutil.copyfile(src, dst)" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "#擷取yun資料集的圖片範圍\n", - "original_0dataset_dir = 'yun'\n", - "fnames = ['yun.{}.png'.format(i) for i in range(20)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(train_y_dir, fname)\n", - " shutil.copyfile(src, dst)\n", - "\n", - "fnames = ['yun.{}.png'.format(i) for i in range(45, 54)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(validation_y_dir, fname)\n", - " shutil.copyfile(src, dst)\n", - "\n", - "fnames = ['yun.{}.png'.format(i) for i in range(54, 59)]\n", - "for fname in fnames:\n", - " src = os.path.join(original_0dataset_dir, fname)\n", - " dst = os.path.join(test_y_dir, fname)\n", - " shutil.copyfile(src, dst)" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "12" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "len(test_y_dir)" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Using TensorFlow backend.\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Model: \"sequential_1\"\n", - "_________________________________________________________________\n", - "Layer (type) Output Shape Param # \n", - "=================================================================\n", - "conv2d_1 (Conv2D) (None, 98, 98, 32) 896 \n", - "_________________________________________________________________\n", - "max_pooling2d_1 (MaxPooling2 (None, 49, 49, 32) 0 \n", - "_________________________________________________________________\n", - "conv2d_2 (Conv2D) (None, 47, 47, 64) 18496 \n", - "_________________________________________________________________\n", - "max_pooling2d_2 (MaxPooling2 (None, 23, 23, 64) 0 \n", - "_________________________________________________________________\n", - "conv2d_3 (Conv2D) (None, 21, 21, 128) 73856 \n", - "_________________________________________________________________\n", - "max_pooling2d_3 (MaxPooling2 (None, 10, 10, 128) 0 \n", - "_________________________________________________________________\n", - "conv2d_4 (Conv2D) (None, 8, 8, 128) 147584 \n", - "_________________________________________________________________\n", - "max_pooling2d_4 (MaxPooling2 (None, 4, 4, 128) 0 \n", - "_________________________________________________________________\n", - "flatten_1 (Flatten) (None, 2048) 0 \n", - "_________________________________________________________________\n", - "dense_1 (Dense) (None, 512) 1049088 \n", - "_________________________________________________________________\n", - "dense_2 (Dense) (None, 3) 1539 \n", - "=================================================================\n", - "Total params: 1,291,459\n", - "Trainable params: 1,291,459\n", - "Non-trainable params: 0\n", - "_________________________________________________________________\n" - ] - } - ], - "source": [ - "#建立cnn\n", - "from keras import layers\n", - "from keras import models\n", - "\n", - "model = models.Sequential()\n", - "model.add(layers.Conv2D(32, (3, 3), activation='relu',\n", - " input_shape=(100, 100, 3)))\n", - "model.add(layers.MaxPooling2D((2, 2)))\n", - "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n", - "model.add(layers.MaxPooling2D((2, 2)))\n", - "model.add(layers.Conv2D(128, (3, 3), activation='relu'))\n", - "model.add(layers.MaxPooling2D((2, 2)))\n", - "model.add(layers.Conv2D(128, (3, 3), activation='relu'))\n", - "model.add(layers.MaxPooling2D((2, 2)))\n", - "model.add(layers.Flatten())\n", - "model.add(layers.Dense(512, activation='relu'))\n", - "model.add(layers.Dense(3, activation='softmax'))\n", - "\n", - "model.summary()" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "#優化器\n", - "from keras import optimizers\n", - "\n", - "model.compile(loss='categorical_crossentropy',\n", - " optimizer=optimizers.RMSprop(lr=1e-4),\n", - " metrics=['acc'])" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Found 60 images belonging to 3 classes.\n", - "Found 27 images belonging to 3 classes.\n" - ] - } - ], - "source": [ - "#ImageDataGenerator套件label圖片\n", - "from keras.preprocessing.image import ImageDataGenerator\n", - "import numpy as np\n", - "train_datagen = ImageDataGenerator(rescale=1./255)\n", - "test_datagen = ImageDataGenerator(rescale=1./255)\n", - "\n", - "train_generator = train_datagen.flow_from_directory(\n", - " train_dir,\n", - " target_size=(100, 100),\n", - " batch_size=20,\n", - " class_mode='categorical')\n", - "\n", - "validation_generator = test_datagen.flow_from_directory(\n", - " validation_dir,\n", - " target_size=(100, 100),\n", - " batch_size=20,\n", - " class_mode='categorical')" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "data batch shape: (20, 100, 100, 3)\n", - "labels batch shape: (20, 3)\n" - ] - } - ], - "source": [ - "for data_batch, labels_batch in train_generator:\n", - " print('data batch shape:', data_batch.shape)\n", - " print('labels batch shape:', labels_batch.shape)\n", - " break" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1/30\n", - "100/100 [==============================] - 94s 940ms/step - loss: 0.6316 - acc: 0.7760 - val_loss: 0.8940 - val_acc: 0.7778\n", - "Epoch 2/30\n", - "100/100 [==============================] - 93s 930ms/step - loss: 0.4135 - acc: 0.8530 - val_loss: 0.6577 - val_acc: 0.7037\n", - "Epoch 3/30\n", - "100/100 [==============================] - 94s 941ms/step - loss: 0.2813 - acc: 0.8860 - val_loss: 0.6228 - val_acc: 0.6296\n", - "Epoch 4/30\n", - "100/100 [==============================] - 93s 933ms/step - loss: 0.1676 - acc: 0.9280 - val_loss: 1.7445 - val_acc: 0.5926\n", - "Epoch 5/30\n", - "100/100 [==============================] - 94s 939ms/step - loss: 0.1259 - acc: 0.9370 - val_loss: 1.0945 - val_acc: 0.5926\n", - "Epoch 6/30\n", - "100/100 [==============================] - 94s 942ms/step - loss: 0.1062 - acc: 0.9405 - val_loss: 2.6368 - val_acc: 0.5926\n", - "Epoch 7/30\n", - "100/100 [==============================] - 96s 960ms/step - loss: 0.0993 - acc: 0.9390 - val_loss: 1.2031 - val_acc: 0.5926\n", - "Epoch 8/30\n", - "100/100 [==============================] - 95s 954ms/step - loss: 0.0951 - acc: 0.9305 - val_loss: 2.8705 - val_acc: 0.5926\n", - "Epoch 9/30\n", - "100/100 [==============================] - 96s 957ms/step - loss: 0.0892 - acc: 0.9385 - val_loss: 2.4154 - val_acc: 0.5926\n", - "Epoch 10/30\n", - "100/100 [==============================] - 97s 966ms/step - loss: 0.0860 - acc: 0.9365 - val_loss: 4.6667 - val_acc: 0.5926\n", - "Epoch 11/30\n", - "100/100 [==============================] - 97s 970ms/step - loss: 0.0860 - acc: 0.9365 - val_loss: 2.0964 - val_acc: 0.5926\n", - "Epoch 12/30\n", - "100/100 [==============================] - 97s 973ms/step - loss: 0.0844 - acc: 0.9365 - val_loss: 4.6467 - val_acc: 0.5926\n", - "Epoch 13/30\n", - "100/100 [==============================] - 97s 969ms/step - loss: 0.0842 - acc: 0.9365 - val_loss: 0.8492 - val_acc: 0.5926\n", - "Epoch 14/30\n", - "100/100 [==============================] - 97s 972ms/step - loss: 0.0811 - acc: 0.9345 - val_loss: 6.7441 - val_acc: 0.5926\n", - "Epoch 15/30\n", - "100/100 [==============================] - 97s 971ms/step - loss: 0.0801 - acc: 0.9380 - val_loss: 1.3565 - val_acc: 0.5926\n", - "Epoch 16/30\n", - "100/100 [==============================] - 97s 973ms/step - loss: 0.0775 - acc: 0.9315 - val_loss: 5.6226 - val_acc: 0.5926\n", - "Epoch 17/30\n", - "100/100 [==============================] - 98s 983ms/step - loss: 0.0816 - acc: 0.9315 - val_loss: 5.0703 - val_acc: 0.5926\n", - "Epoch 18/30\n", - "100/100 [==============================] - 98s 984ms/step - loss: 0.0746 - acc: 0.9330 - val_loss: 7.8723 - val_acc: 0.5926\n", - "Epoch 19/30\n", - "100/100 [==============================] - 98s 983ms/step - loss: 0.0747 - acc: 0.9360 - val_loss: 2.2070 - val_acc: 0.5926\n", - "Epoch 20/30\n", - "100/100 [==============================] - 98s 979ms/step - loss: 0.0756 - acc: 0.9320 - val_loss: 8.8935 - val_acc: 0.5926\n", - "Epoch 21/30\n", - "100/100 [==============================] - 98s 983ms/step - loss: 0.0775 - acc: 0.9350 - val_loss: 2.1388 - val_acc: 0.5926\n", - "Epoch 22/30\n", - "100/100 [==============================] - 98s 984ms/step - loss: 0.0734 - acc: 0.9360 - val_loss: 4.9511 - val_acc: 0.5926\n", - "Epoch 23/30\n", - "100/100 [==============================] - 98s 982ms/step - loss: 0.0729 - acc: 0.9335 - val_loss: 1.6979 - val_acc: 0.5926\n", - "Epoch 24/30\n", - "100/100 [==============================] - 99s 988ms/step - loss: 0.0718 - acc: 0.9325 - val_loss: 3.6615 - val_acc: 0.5926\n", - "Epoch 25/30\n", - "100/100 [==============================] - 98s 983ms/step - loss: 0.0729 - acc: 0.9375 - val_loss: 6.6448 - val_acc: 0.5926\n", - "Epoch 26/30\n", - "100/100 [==============================] - 98s 984ms/step - loss: 0.0721 - acc: 0.9355 - val_loss: 6.3982 - val_acc: 0.5926\n", - "Epoch 27/30\n", - "100/100 [==============================] - 98s 980ms/step - loss: 0.0711 - acc: 0.9340 - val_loss: 12.2808 - val_acc: 0.5926\n", - "Epoch 28/30\n", - "100/100 [==============================] - 98s 980ms/step - loss: 0.0721 - acc: 0.9365 - val_loss: 7.8047 - val_acc: 0.5926\n", - "Epoch 29/30\n", - "100/100 [==============================] - 98s 980ms/step - loss: 0.0704 - acc: 0.9310 - val_loss: 9.7293 - val_acc: 0.5926\n", - "Epoch 30/30\n", - "100/100 [==============================] - 98s 982ms/step - loss: 0.0737 - acc: 0.9360 - val_loss: 6.4731 - val_acc: 0.5926\n" - ] - } - ], - "source": [ - "#epoch訓練30\n", - "history = model.fit_generator(\n", - " train_generator,\n", - " steps_per_epoch=100,\n", - " epochs=30,\n", - " validation_data=validation_generator,\n", - " validation_steps=50)" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [], - "source": [ - "model.save('train1.h5')" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtcAAAEjCAYAAAD5ZS3PAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3zV5fXA8c/JIpOEDJCw90amoCDDCYqCaFUUK1bKT+usxaqtdbaO1oFWa4sDF4ooDlSstoqiULaIbJAZwkgCJCGDrPP747mBEDNuyL1ZnPfrdV/33u94vucGTc59vud5HlFVjDHGGGOMMdUXUNsBGGOMMcYY01BYcm2MMcYYY4yPWHJtjDHGGGOMj1hybYwxxhhjjI9Ycm2MMcYYY4yPWHJtjDHGGGOMj1hybYwxpYjIZyJybW3HURvEmSEiB0VkaW3HY4wx9Y0l18aYektEtotIjogc9iSDn4pIq+q2q6qjVfW1E4hHRORWEVkjIlkikiQi74pIL8/+V0VEReS0Eud0FBEt8f5rEckt+TlE5BwR2V7BddVzvcMisltEnhKRwKrG7zEUOBdoqaqnVXawMcaY41lybYyp7y5S1UigObAP+HstxvIMcBtwKxALdAY+BC4sccwB4M+VtJMF/KmK1z7V83M4G7gK+HUVz0dEgoA2wHZVzTrB840x5qRmybUxpkFQ1VzgPaB78TYRuVBEvheRDBHZJSIPlNgXKiJvikiaiBwSkWUi0syz72sRmVzi2F+LyHoRyRSRdSLSr/T1RaQTcBMwQVW/UtUjqpqtqjNV9bESh74G9BaR4RV8nGeBCSLS8QR+DhuAb4GenrgSRWSOiKSIyDYRubVEzA+IyHuen0MGcD3wEnC6pxf8wRKff4uIHBCRuSKSWKINFZGbRGQzsLnEtt+IyGbPz+xhEekgIv/z/FvMFpEQz7FNROQTT3wHPa9blmj/a8/5Cz1tfSEi8SX2DxWRRZ5/w10iMsmzvZGIPCEiO0Vkn4j8U0TCqvrzNMaYqrLk2hjTIIhIOHAFsLjE5izgl0AMrvf4RhEZ59l3LRANtALigBuAnDLa/QXwgKedxsDFQFoZIZwNJKlqZXXK2cAjwF8qOGY38KLnulUiIt2BM4HvRSQA+Bj4AWjhifF2ETm/xCljcV9KYoDXcT+H/6lqpKreLyJnAY8Cl+PuDuwAZpW67DhgECW+2ACjgP7AYOD3wHTgatzPuycwwXNcADAD12PeGvdv8Fyp9q8CrgOaAiHAVM9nbQ18hrtbkQD0AVZ5znkcd+egD9DR8/nvq+hnZ4wxvmDJtTGmvvtQRA4BGbha4b8V71DVr1X1R1UtUtXVwNtAcY9xPi6p7qiqhaq6QlUzymh/MvBXVV2mzhZV3VHGcXHAHi9j/hfQWkRGV3DMo8BFItLDyzZXishBXDL9Ei5hHQgkqOpDqpqnqltxSfuVJc77n6p+6PkZ/ezLBS4hfkVVV6rqEeAeXM9225KxquqBUuc/rqoZqroWWAN8oapbVTUdlxD3BVDVNFWd4+nlz8R96Sjdqz9DVTd52p+NS5iLY/uvqr6tqvmetlaJiODKYn7riSsT94XmSowxxs+sPs4YU9+NU9X/egbwjQW+EZHuqrpXRAYBj+F6SkOARsC7nvPewPWizhKRGOBN4I+qml+q/VbAT17EkYbr2a2Uqh4RkYeBhznWg1v6mBQReQ54CHjBi2b7qeqWkhtEpA2Q6PnyUSwQVzZSbFcl7SYCK0vEdVhE0nA9wdsraGNfidc5Zbw/xRNjOPA0rqe7iWd/lIgEqmqh5/3eEudmA5Ge1+X92yQA4cAKl2cDILjPbowxfmU918aYBsHT+/w+UIib8QLgLWAu0EpVo4F/4pIsPD2dD6pqd+AMYAyu9KO0XUAHL0L4EmgpIgO8DHkGrizlkgqO+RswEldecSJ2AdtUNabEI0pVLyhxjJZ3skcyrmQDABGJwPXS765CGxX5HdAFGKSqjYFhxZfy4tzy/m1ScQl8jxKfO9oz4NMYY/zKkmtjTIMgzlhc7+d6z+Yo4ICq5nqmv7uqxPEjRaSXp8c7A1cmUli6XVyJxVQR6e+5RkdPj/BxVHUz8A/gbREZISIhnkGTV4rI3WUcX4Crqb6rvM+kqoeAJ3E1yydiKZAhIneJSJiIBIpITxEZWIU23gKuE5E+ItIIV16xRFW3n2BMpUXhEuFDIhIL3F+Fc2cC54jI5SISJCJxItJHVYtw5S9Pi0hTABFpUarW3Bhj/MKSa2NMffexiBzGJch/Aa711PkC/AZ4SEQycYPZZpc47xTcQL4MXDL+Da405Diq+q6n3beATNzUerHlxHIrbjDe88AhXMnCJbg66LK8TeV12s9QdtJfKU9ZxUW4GuVtuB7dl3A95t628SVuWsA5nlg74Nva5WlAmCe2xcC/qxDbTuACXO/3AdxgxlM9u+8CtgCLPTOh/BfXQ26MMX4lqtW5m2eMMcYYY4wpZj3XxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXpl4Rka9F5KCINKrtWIwxxtQ+EdkuIufUdhzGFLPk2tQbItIWOBNQ4OIavG5QTV3LGGOMMfWbJdemPvklsBh4Fbi2eKOIhInIkyKyQ0TSReQ7EQnz7BsqIotE5JCI7BKRSZ7tX4vI5BJtTBKR70q8VxG5SUQ2A5s9257xtJEhIitE5MwSxweKyB9E5CcRyfTsbyUiz4vIkyU/hIh8LCK3++MHZIwxxhGRX4vIFhE5ICJzRSTRs11E5GkR2e/5m7FaRHp69l0gIus8v8d3i8jU2v0Upj6y5NrUJ78EZnoe54tIM8/2J4D+wBlALPB7oEhEWgOfAX8HEoA+wKoqXG8cMAjo7nm/zNNGLPAW8K6IhHr23QFMAC4AGgO/ArKB14AJIhIAICLxwNnA21X54MYYY7wnImcBjwKXA82BHcAsz+7zgGFAZyAGuAJI8+x7Gfg/VY0CegJf1WDYpoGw5NrUCyIyFGgDzFbVFcBPwFWepPVXwG2qultVC1V1kaoeAa4G/quqb6tqvqqmqWpVkutHVfWAquYAqOqbnjYKVPVJoBHQxXPsZOBeVd2ozg+eY5cC6biEGuBK4GtV3VfNH4kxxpjyXQ28oqorPX8P7gFO95QX5gNRQFdAVHW9qu7xnJcPdBeRxqp6UFVX1kLspp6z5NrUF9cCX6hqquf9W55t8UAoLtkurVU52721q+QbEfmdiKz33EY8BER7rl/ZtV4DJnpeTwTeqEZMxhhjKpeI660GQFUP43qnW6jqV8BzwPPAPhGZLiKNPYdeirsDuUNEvhGR02s4btMAWHJt6jxP/fTlwHAR2Ssie4HfAqfibvflAh3KOHVXOdsBsoDwEu9PKeMYLRHDmcBdnjiaqGoMrkdavLjWm8BYETkV6AZ8WM5xxhhjfCMZd7cTABGJAOKA3QCq+qyq9gd64MpD7vRsX6aqY4GmuN/Vs2s4btMAWHJt6oNxQCGu9rmP59EN+BZXh/0K8JSIJHoGFp7umapvJnCOiFwuIkEiEicifTxtrgLGi0i4iHQErq8khiigAEgBgkTkPlxtdbGXgIdFpJNnsExvEYkDUNUkXL32G8Cc4jITY4wxPhMsIqHFD1xSfJ2I9PH8PXgEWKKq20VkoIgMEpFgXEdLLlAoIiEicrWIRKtqPpCB+9tjTJVYcm3qg2uBGaq6U1X3Fj9wt/WuBu4GfsQlsAeAx4EAVd2Ju733O8/2VbjeboCngTxgH65sY2YlMXyOGxy5CXerMZfjy0aewv0y/wL3C/llIKzE/teAXlhJiDHG+MM8IKfE40zgT8AcYA/uzuKVnmMbAy8CB3G/z9NwA+MBrgG2i0gGcAPHSvqM8ZqoauVHGWOqRUSG4cpD2qpqUW3HY4wxxhj/sJ5rY/zMc+vxNuAlS6yNMcaYhs2Sa2P8SES6AYdwAy+n1XI4xhhjjPEzKwsxxhhjjDHGR6zn2hhjjDHGGB8Jqu0AfCU+Pl7btm1b22EYYwwrVqxIVdWE2o6jIbLf9caYuqCi3/MNJrlu27Yty5cvr+0wjDEGEdlR+VHmRNjvemNMXVDR73m/loWIyCgR2SgiW0Tk7jL2txGRL0VktYh8LSItS+y7VkQ2ex7X+jNOY4wxxhhjfMFvybWIBALPA6NxK+tNEJHupQ57AnhdVXsDDwGPes6NBe4HBgGnAfeLSBN/xWqMMcYYY4wv+LPn+jRgi6puVdU8YBYwttQx3YEvPa/nl9h/PvAfVT2gqgeB/wCj/BirMcYYY4wx1ebPmusWHL88dBKuJ7qkH4BLgWeAS4AoEYkr59wW/gvVGGNMfZWfn09SUhK5ubm1HYqpgtDQUFq2bElwcHBth2KMT/kzuZYytpWeVHsq8JyITAIWALuBAi/PRUSmAFMAWrduXZ1YjTHG1FNJSUlERUXRtm1bRMr682HqGlUlLS2NpKQk2rVrV9vhGONT/iwLSQJalXjfEkgueYCqJqvqeFXtC/zRsy3dm3M9x05X1QGqOiAhwWa9MsaYk1Fubi5xcXGWWNcjIkJcXJzdbTANkj+T62VAJxFpJyIhwJXA3JIHiEi8iBTHcA/wiuf158B5ItLEM5DxPM82Y4wpl6qyPTWL7alZPmlrmw/aMTXDEuv6x/7NTJ3w03zIOejTJv1WFqKqBSJyMy4pDgReUdW1IvIQsFxV5wIjgEdFRHFlITd5zj0gIg/jEnSAh1T1gL9iNcbUT0VFyqb9mSzdduDoY3/mEQBObRnNpf1bclHvRJpEhHjd5vbULOb+kMxHq3azLTWLxfecTdPGof76CMYYY2rLntXw9pXQ4xK45J8+a9avi8io6jxgXqlt95V4/R7wXjnnvsKxnmxjjCG/sIi1yRks23aAJdsOsGz7AdJz8gFoHh3K6R3iOK1dLDl5hcxZuZv7PlrLw5+s46yuTbm0X0tGdm1KcODPb9jtz8zl09V7+HBVMj/sOgTAoHaxXD+0PeGNGsxaWzVKRF4BxgD7VbVnBccNBBYDV3j+JtQ7aWlpnH322QDs3buXwMBAiksVly5dSkhI5V/urrvuOu6++266dOni1TVfeukl1qxZw7Rp0048cGNOZjkHYfY1ENYEzn3Ip03bXw1jTJ2TX1jEjrQsNu87zKZ9h9m0P5Mt+w6zNfUw+YVubHO7+AhG9TiF09rFclq7WFo2CTvuNvPkM9uzLjmDOSuT+GjVbj5fu4/YiBAuPjWRy/q3pHVcOJ+v2cvcH5JZuCWVIoXuzRvzhwu6MqZ3IokxYbX18RuKV4HngNfLO8CzHsLj1POyv7i4OFatWgXAAw88QGRkJFOnTj3uGFVFVQkIKLsac8aMGX6P0xjjUVQE70+B9N1w3TyIbOrT5i25NsZUS0FhEYX6s8l8jqMKOXmFHD5SQEZuPodzCzh8xD0yc4sf+ew4kP2zJFoEWjUJp3OzSM7q1pQeiY05rV0sTaMqL9XontiY7onduXt0VxZsSmHOyiTeWrKTVxdtJzBAKCxSWseGc9PIjlx8aiKdmkX55GdiQFUXiEjbSg67BZgDDPR7QLVgy5YtjBs3jqFDh7JkyRI++eQTHnzwQVauXElOTg5XXHEF993nbuYOHTqU5557jp49exIfH88NN9zAZ599Rnh4OB999BFNm3r3x//NN9/k8ccfR1W5+OKLeeSRRygoKOC6665j1apVqCpTpkzh1ltv5emnn+bFF18kODiYXr168eabb/rzx2FM3bHgb7D5C7jgCWh1ms+bt+TaGFNl21Kz+HL9Pr7asJ+l2w5QUFRxcu2NwAChRUwYnZtFMrJrUzo3i6Rzsyg6JEQSFhJYrbaDAwM4u1szzu7WjPTsfD5enUzSwRzO79GMPq1ibGBVLRCRFrj1Dc6ikuS6KtOuPvjxWtYlZ/goSqd7YmPuv6jHCZ27bt06ZsyYwT//6eo5H3vsMWJjYykoKGDkyJFcdtlldO9+/OLF6enpDB8+nMcee4w77riDV155hbvvvrvSayUlJXHvvfeyfPlyoqOjOeecc/jkk09ISEggNTWVH3/8EYBDh1zp01//+ld27NhBSEjI0W3GNHib/wNfPwq9r4SBk/1yCUuujWlADmblMWdlEltTsxjcPo4zO8ZXaTBfefILi1i2/QBfrd/PVxv2s9Uzi0aXZlFcN6QtMeGVXyM8JJDIRkFEhQYTFRpEZKMgIkODiAoNIqpRMKHBATWS5EaHBzNxcBu/X8dUahpwl6oWVvbvrqrTgekAAwYMqP43uRrUoUMHBg489t3h7bff5uWXX6agoIDk5GTWrVv3s+Q6LCyM0aNHA9C/f3++/fZbr661ZMkSzjrrLOLj4wG46qqrWLBgAXfddRcbN27ktttu44ILLuC8884DoEePHkycOJGxY8cybtw4X3xcY+q2g9thzmRo1gPGPO1ujfqBJdfG1HOqysqdh5i5eAef/LiHvIIiIkICeWvJTkSgd4tohnVOYFjnBPq2iiGojAF9pds7lJ3ProPZbNp3mPkb97NgYwqZRwoICQzg9A5xTBrSlpFdmtIqNryGPqVpgAYAszyJdTxwgYgUqOqH1Wn0RHuY/SUiIuLo682bN/PMM8+wdOlSYmJimDhxYpnzPJccABkYGEhBQYFX19JyyrPi4uJYvXo1n332Gc8++yxz5sxh+vTpfP7553zzzTd89NFH/PnPf2bNmjUEBlbvLpExdVZ+DrxzDaBwxRsQ4r+/X5ZcG1NPHT5SwIff72bmkp2s35NBZKMgrhjQiqsGtaZzsyh+3J3ONxtTWLA5hefnb+HvX20hKjSIIR3iGdY5gd4to9mXkcuuA9nsOpjDrgPZ7DyQTdLBHA4fOfbHPCGqERf2bs5ZXZsypGM8ETZ7hvEBVT26LJ+IvAp8Ut3Euq7LyMggKiqKxo0bs2fPHj7//HNGjRrls/YHDx7MnXfeSVpaGtHR0cyaNYupU6eSkpJCaGgov/jFL2jXrh033HADhYWFJCUlcdZZZzF06FBmzpxJdnY2UVE27sA0QKrw6e9g72q4ajbEtvfr5eyvpDH1zLrkDGYu2cGH3+8mK6+Q7s0b88glvbi4TyKRJRLfPq1i6NMqhtvO6UR6dj4Lf0plwaYUFmxK4d9r9x7XZmhwAK2ahNMqNpzB7eNo2SSMVrHhtI2LoFPTSAICrCbZVI2IvI1byyBeRJKA+4FgAFX13YSy9Ui/fv3o3r07PXv2pH379gwZMqRa7b388su8996x2QuXL1/OQw89xIgRI1BVLrroIi688EJWrlzJ9ddfj6oiIjz++OMUFBRw1VVXkZmZSVFREXfddZcl1qZu2L8BvvgjjP4rxHXwTZsrZsCqmTD8Luh8vm/arICUdxupvhkwYIAuX768tsMwxueOFBSybNtBFmx2ifGGvZk0CgpgTO9EJg5uXeUBearKlv1uirvmMaG0ahJOfGSIDerzIRFZoaoDajuOhqis3/Xr16+nW7dutRSRqQ77tzM/8841sH4uNOsJ1/+n+uUbScvhlVHQfrjrtQ7wTelTRb/nrefamDpGVdmamnW0l3nx1gPk5BcSHCgMaBPLfWO6M75fC68GEZZFROjULMqmnTPGGFO3pG6G9R9Du+GwbQHMmwpjnz/xgYdZqTD7l9C4OYx/0WeJdWUsuTamGgoKi1i/J5O9GbkczM7jUHYeB7PzOZSdx4GsY68PZecTGCAlZskIJqpRkGf2DDdrRmSjILamZvHNxhR2H8oBoG1cOJcPaMmwzgkMbh9n9c7GGGMarkXPQlAjuPQlWPYSfPM4tBoE/a+telsFR+C96yA7Da7/AsJjfR9vOewvtTFVUFSkbNyXyaKf0li0JZUl2w4cN/gPIDhQiAkPoUl4MDHhIbSLjyAmLIQi1aOLpqTn5LP7YDaZnsVUsvMKAYgICeSMjvHcMKIDwzsl0DrOZuMwxhhzEsjYAz/Mgr7XuBUTh98FSctg3p3QvDck9vW+rSOH4Z2Jrvd73D+h+an+i7sMllwbUwFVZUdaNgt/SmXRT2ks/imNtKw8wC2/fXGfRE5vH0fbuAhiwoNpEhFCREhgleuXCwqLyDpSSHijQIIrmSrPGGOMaXAW/wOKCuCMW9z7gEAY/xL8a5gr7ZjyjXe9zzkHYeYvYPcKGPcC9Jng37jLYMm1MaWk5+SzcIubWePbzalHSzSaNW7E8M4JnNExntM7xNEiJsxn1wwKDCA63JJqY4wxJ6GcQ7B8BvS4BGLbHdseEQeXv+YGJH5wA0yYBQEV/K3M3AdvjofUTXD569DtIv/HXgZLrs1Jr7BIj5sTetWuQxQWKVGNgjijYxw3jOjAGR3iaB8fYTNqGGOMMb62/GXIy4Qht/98X8sBMOpRN7jxuydh2J1lt3FoJ7w+FjL3ullBOoz0b8wVsK4yc9LJzM3nx6R0Zi/bxc1vraT/n//DuOcXMu3LTRQUFvGbER1494bTWXnfufzrmgFcM7gNHRIiLbE2xpRpxIgRfP7558dtmzZtGr/5zW8qPC8yMhKA5ORkLrvssnLbrmya2WnTppGdnX30/QUXXMChQ4e8Cb1CDzzwAE888US12zGmQvk5sPif0OFsV1tdloGTodcv4Ku/wE/zf74/ZZPr3c5Og19+VKuJNVjPtWmgjhQUsjMtm62pWWxLzWJbiuc5LYuUzCNHj0uIasTZXZsxrHM8Z3ZKIDbixKa3M8acvCZMmMCsWbM4//xji1PMmjWLv/3tb16dn5iYeNxiMFU1bdo0Jk6cSHi4GwA9b968E27LmBq36i3I2g9Df1v+MSJw0TOwdw3MuR7+71uIbuH27fkB3hjvjpn0KZzSq2biroAl16beKSgsYn/mEfak57AnPZc9h3Ldc/H79Bz2Zx6h5PpI8ZFu1o6RXRJoFx9Ju/gIOjaNsB5pY0y1XXbZZdx7770cOXKERo0asX37dpKTkxk6dCiHDx9m7NixHDx4kPz8fP785z8zduzY487fvn07Y8aMYc2aNeTk5HDdddexbt06unXrRk5OztHjbrzxRpYtW0ZOTg6XXXYZDz74IM8++yzJycmMHDmS+Ph45s+fT9u2bVm+fDnx8fE89dRTvPLKKwBMnjyZ22+/ne3btzN69GiGDh3KokWLaNGiBR999BFhYd6NIymrzaysLC6//HKSkpIoLCzkT3/6E1dccQV33303c+fOJSgoiPPOO896ws3xCgvc9Hst+kPboRUfGxIBV7wB00fAu9fCpHlu0OJbl0NotOux9tWKjtXk1+RaREYBzwCBwEuq+lip/a2B14AYzzF3q+o8EWkLrAc2eg5drKo3+DNWUzfl5BWybk86q5OKH4fYlppFUamFRcNDAmkeHUpiTBidmibQPCaM9vERtIuPoG18BNFhwbXzAYwxNeuzu2Hvj75t85ReMPqxcnfHxcVx2mmn8e9//5uxY8cya9YsrrjiCkSE0NBQPvjgAxo3bkxqaiqDBw/m4osvLvdL/QsvvEB4eDirV69m9erV9OvX7+i+v/zlL8TGxlJYWMjZZ5/N6tWrufXWW3nqqaeYP38+8fHxx7W1YsUKZsyYwZIlS1BVBg0axPDhw2nSpAmbN2/m7bff5sUXX+Tyyy9nzpw5TJw4sdIfRXltbt26lcTERD799FMA0tPTOXDgAB988AEbNmxARHxSqmI8jhx2M2IMvB56lV1SVC+s/wgObodzH/ZuoZj4Tm5RmXevhXeuhm3fQnRL+OWH7rmO8FtyLSKBwPPAuUASsExE5qrquhKH3QvMVtUXRKQ7MA9o69n3k6r28Vd8puaoKpv2HSb18BECA4TgQCEoIICgEs/BnueUzCOs3p3Oj0mHWJ2Uzub9hyn0ZNLNGjeiV4sYLujVnMSYME6JDiUx2j03Dg2yHmhjTK0pLg0pTq6Le3ZVlT/84Q8sWLCAgIAAdu/ezb59+zjllFPKbGfBggXceuutAPTu3ZvevY/VoM6ePZvp06dTUFDAnj17WLdu3XH7S/vuu++45JJLiIiIAGD8+PF8++23XHzxxbRr144+fdyf2P79+7N9+3avPmd5bY4aNYqpU6dy1113MWbMGM4880wKCgoIDQ1l8uTJXHjhhYwZM8araxgvrJ8LOxdB8vfQtDs0617bEVWdKnw3DeI6Qdcq/LfRYxzsugkWP++++E78ACIT/BfnCfBnz/VpwBZV3QogIrOAsUDJ5FqBxp7X0UCyH+MxNehgVh7feqazW7Aphf0l6py9ERsRQu+W0ZzXvRm9WsbQu2U0zRqH+ilaY0yDUUEPsz+NGzeOO+64g5UrV5KTk3O0x3nmzJmkpKSwYsUKgoODadu2Lbm5uRW2VVZHwbZt23jiiSdYtmwZTZo0YdKkSZW2o6rl7mvUqNHR14GBgceVn5xIm507d2bFihXMmzePe+65h/POO4/77ruPpUuX8uWXXzJr1iyee+45vvrqK6+uYyrxwyyIbgWFea4X99fzoVFkbUdVNT99BXtXw8V/r3h6vbKc+yAk9oHO57uSkDrGn8l1C2BXifdJwKBSxzwAfCEitwARwDkl9rUTke+BDOBeVf229AVEZAowBaB169a+i9xUWUFhEd/vOnQ0mV69Ox1ViA4LZmineIZ3SqBtfAQFhUXkFymFRUXkFyoFhUpBUdHR58ahwfRqGU2LmDDriTbG1BuRkZGMGDGCX/3qV0yYcGzRivT0dJo2bUpwcDDz589nx44dFbYzbNgwZs6cyciRI1mzZg2rV68GICMjg4iICKKjo9m3bx+fffYZI0aMACAqKorMzMyflYUMGzaMSZMmcffdd6OqfPDBB7zxxhvV+pzltZmcnExsbCwTJ04kMjKSV199lcOHD5Odnc0FF1zA4MGD6dixY7WubTzSd7uVB0fcDW3OcNPPffJbGD/du9KKumLhNIhqDr2vqPq5gcHQ+3Lfx+Qj/kyuy/oXLv2VdwLwqqo+KSKnA2+ISE9gD9BaVdNEpD/woYj0UNWM4xpTnQ5MBxgwYED5XyZaUjEAACAASURBVNGNX+TmF/L1xhQ+/iGZBZtTyMwtIECgT6sYbj+7M8M6x9O7ZQyBAfXof3ZjjDlBEyZMYPz48cyaNevotquvvpqLLrqIAQMG0KdPH7p27VphGzfeeCPXXXcdvXv3pk+fPpx22mkAnHrqqfTt25cePXrQvn17hgwZcvScKVOmMHr0aJo3b878+cemKevXrx+TJk062sbkyZPp27ev1yUgAH/+85+ZNm3a0fdJSUlltvn5559z5513EhAQQHBwMC+88AKZmZmMHTuW3NxcVJWnn37a6+uaCvw4G1CXXMa2hxH3wPy/QNsh0H9SbUfnnd0r3BeEcx+GoEaVH1/PSEW3jarVsEuWH1DV8z3v7wFQ1UdLHLMWGKWquzzvtwKDVXV/qba+BqaqarmTfQ4YMEArmwvUVF9hkbJkaxofrUpm3po9ZOYWEBcRwjndmjG8SwJDOsQTHW6DB83JTURWqOqA2o6jISrrd/369evp1q1bLUVkqsP+7apIFf4x2JVCXP+F21ZUCG9eCjsWwa+/rBNT0VXqnWtg2zdw+xoIbVz58XVQRb/n/dlzvQzoJCLtgN3AlcBVpY7ZCZwNvCoi3YBQIEVEEoADqlooIu2BTsBWP8ZqKqCqrNmdwYerdvPxD8nszzxCREgg5/c8hbF9WjCkQxxBgbYekTHGGONXe36AlA0wpsRdgIBAGP8i/OtMmH0tTPm6biesqVtg/cdw5h11O85q8FtyraoFInIz8Dlumr1XVHWtiDwELFfVucDvgBdF5Le4kpFJqqoiMgx4SEQKgELgBlU94K9YG5K96bms3HmQDXsyOHykkJz8Qo7ku+fco89F5OYXkldQRHBgAKEhgYQGBRAWEkhoUKB7Dg4gNDgQQfh64362pmYRHCiM6NKUcX1acFbXpoSFBNb2xzXGGGNOHqvfgcAQ6HHJ8dsjE+DSl+G1MfDxbXDZK3W3/nrRM64UZFDDnWHZr/Ncq+o83PR6JbfdV+L1OmBIGefNAeb4M7aGIK+giLXJ6azceYiVOw/y/Y6DJKe70eMBAuEhQYQGu0Q5LDjwaPLcOCyYplGNCAkKIL+wiNz8InLyCzmYlXc0+c7JLyQ3r5AjhUX0b92EKcPaM7pncyv5MMbUSapqg6DrGX+VpTZYhQXw47tuhoywJj/f33YInHUvfPmQez1wcs3HWFJhAeQd9jyy3HNWmpvppO81ENm0duPzI1uhsY5SVTKPFHAoK5+D2XkczM7jULZ7nXwoh5U7D/Hj7nTyCooAaBETRr82TZjcugn92jShe/PGhARZqYYxpuELDQ0lLS2NuLg4S7DrCVUlLS2N0FCbYtVrP30FWSlw6oTyjxnyW9jxP/j3PdBigJuuriaowndPwYrXXBJ95DAUljMFb0AwnHFLzcRVSyy5rgNUleU7DjJnRRIrdhw8mkgXlF6G0CMkKIBeLaK59vQ29PMk0zYHtDHmZNWyZUuSkpJISUmp7VBMFYSGhtKyZd1ZVa/OWz0LwmKh47nlHxMQAJf8y9Vfv3st/N8C/88DXVQIn94BK16FdsMhrqNbqrxRlHsOiTz23CgSGreA2Hb+jamWWXJdi3YdyOb9lbt5//skdqRlEx4SyBkd4hnQNpYm4cE0CQ8hxvPcJCKYmPAQmoSHEB0WbNPbGWOMR3BwMO3aNew/1uYkl5sBGz6FvhMhKKTiYyPiXM31jAvgo5vh8tf9V3+dnwNzJsOGT2DoHXD2fXW31rsGWXJdww4fKWDe6j3MWZnEkm1ujObp7eO45axOjO55ChGN7J/EGGOMMSWs+wgKcisuCSmp9WA45374z32w8jX/zH+dfQDengC7lsDov8Kg//P9Neopy+RqgKqyZNsBZi3dyb/X7iU3v4h28RFMPa8z4/q2oGWT8NoO0RhjjDF11ep3ILYDtOjv/Tmn3wIbP4OvH3dJuS8Xa0lPcnNrH9jqesl7jvdd2w2AJdd+lFdQxLwf9/DSd1tZszuDqNAgxvdryaX9WtKvdYwNvDHGNFgi8gowBtivqj3L2H81cJfn7WHgRlX9oQZDNKZ+OLQTtn8LI/9YtZKLgAAYfhe8MQ5WzYQBv/JNPPs3wJvj4UgmTJwD7Yb5pt0GxJJrP0jPzuetpTt5bdF29mbk0iEhgkcu6cX4fi0IDba5oY0xJ4VXgeeA18vZvw0YrqoHRWQ0MB0YVEOxGVN/rJ7tnntfXvVz249ws4Z897Sb/i6wmtPp7lwMb13hesGvm1c/VoOsBZZc+9D21CxmLNzG7OVJ5OQXMrRjPI9e2ovhnRIIsAGIxpiTiKouEJG2FexfVOLtYsCmjTCmNFVXEtL6DGjSturni8Dw38Nbl7s5svuUXii7CjZ8Cu/9CqJbwsT3oUmbE2+rgbPkuppUlcVbDzBj4Tb+s34fQQHC2D4tuH5oO7o1b5jLehpjjI9dD3xW3k4RmQJMAWjdunVNxWRM7UteCamb4KKbTryNTue5HuZvn4TeV7jl0qvqh1nw4Y2Q2BeuetfNSGLKZcn1CUrPyef9lUnMXLKTLfsP0yQ8mJtHduSawW1oanNOG2OMV0RkJC65HlreMao6HVc2woABA2xZP3Py+OEdCGwE3cedeBsiMOxOmP1LWPsB9LqsauenboGPb4c2Q+Cqd9yc1aZCllxX0Q+7DvHm4h18vDqZ3PwiTm0Vw98u682Y3omEhVg9tTHGeEtEegMvAaNVNa224zGmTinMhzXvQZfREBZTvba6XgQJXV3vdY/xbrCjN4oK4aPfuLm1x79oibWXLLn2QnZeAXNXJTNzyU5+3J1OWHAgl/RtwdWD2tCzhZ9XPjLGmAZIRFoD7wPXqOqm2o7HmBqxe6Wbe7rftdDz0oqT3C3/hew0OPXK6l83IADOnArvT4aNn0K3i7w7b/ELbh7rS6ZD4+bVj+MkYcl1Jf75zU88/9UWMo8U0LlZJA+N7cG4vi1oHFrNEbfGGNOAicjbwAggXkSSgPuBYABV/SdwHxAH/MMzLWmBqg6onWiNqQG56fDuJEjf5abWW/QMnPMgdDy77ON/mAXhcdDxHN9cv8cl8PUjsOBv0HVM5dP6pWyCrx6GLhee2EwlJzFLriuQnVfA4//ewGltY5l6fhcGtGlic1MbY4wXVLXCpeRUdTIwuYbCMaZ2qcKnv3OLr0ya556/esjNF91uOJz7oBssWCznkFsApv+k6k+fVywwyC1RPvdm1yve6dzyjy0qdAMYg8NgzNO2pHkVeVl0c3LasDcTVbh+aDsGto21xNoYY4wxVbf6HTcV3oi7oc3p0PsXcPNyGPUY7P0Rpo+Ad69zKx4CrPsQCo/AqVf4No5Tr4ToVvDNX13CX55Ff4fdy+GCJyCqmW9jOAlYcl2BtckZAPSwumpjjDHGnIi0n1yvdZshcObvjm0PagSDb4TbfoBhv4dN/4bnBsK8O2Hl6xDfGRL7+TaWwGAYejskLYVtC8o+Zv8GmP+Iq8vuealvr3+S8GtyLSKjRGSjiGwRkbvL2N9aROaLyPcislpELiix7x7PeRtF5Hx/xlmedcnpxIQHkxhtU+sZY4wxpooK8mDOZAgIgvHTy55jOrQxnPVHuPV7N9Bx2cuwe4Wbk9ofd8z7TITIU1ztdWmFBa4cJCQCLnzKykFOkN+SaxEJBJ4HRgPdgQki0r3UYfcCs1W1L3Al8A/Pud0973sAo3ADXmp8nru1yRn0SGxs5SDGGGOMqbr5f3ELwVz8d7eyYUWiToExT8FNS2HEPTDQT0MSgkNhyK1uUOXOxcfvW/SMi/fCJyGyqX+ufxLwZ8/1acAWVd2qqnnALGBsqWMUKF7GMBpI9rweC8xS1SOqug3Y4mmvxuQXFrFhbybdbZVFY4wxxlTV1q9h4TNuUGL3i70/L76jq82u7tzWFek/CcLjj++93rcO5j/qFqzpOd5/1z4J+DO5bgHsKvE+ybOtpAeAiZ5pmuYBt1ThXERkiogsF5HlKSkpvoobgK0pWeQVFNEj0eqtjTHGGFMFWanw/v9BfCc4/5HajubnQiLg9JvcrCG7V7oFaz68EUKjXa+1qRZ/Jtdl1VKUHpo6AXhVVVsCFwBviEiAl+eiqtNVdYCqDkhISKh2wCWtTU4HoEei9VwbY4wxxkuq8NHNkHMALnul7q5qOHAyhMa4VRu/mwZ7VrmylIj42o6s3vPnPNdJQKsS71tyrOyj2PW4mmpU9X8iEgrEe3muX61NziA0OID2CZE1eVljjDHG1GdLX4RNn7lp9k7pVdvRlC+0sZut5OtH3UwlPS+F7qWrd82J8GfP9TKgk4i0E5EQ3ADFuaWO2QmcDSAi3YBQIMVz3JUi0khE2gGdgKV+jPVn1ian0+WUxgQG2GBGY4wx5qRVkAef/xGeH+xWWFzwBGz6AjKSfz5X9N418MW90Ok8GHRDrYRbJadNgZAoCGsCo8uYPcScEL/1XKtqgYjcDHwOBAKvqOpaEXkIWK6qc4HfAS+KyG9xZR+TVFWBtSIyG1gHFAA3qWqhv2ItI3bWJWcw5tTEmrqkMcYYY+qajD3w7rWwawm0PROSv4e1HxzbHx4HzXq6HupTesF3T7u65bH/qB/T2IXHwjUfuF7siLjajqbB8Ovy56o6DzdQseS2+0q8XgcMKefcvwB/8Wd85Uk6mENGboHVWxtjjDEnq+3fuVUT87Jc7XTxgiq56bBvreul3rsa9q1xpSCFR9z+ie9DpG/HgflVq4G1HUGD49fkur46ujKjzRRijDHGnFxUYfE/4Is/QWw7uHYuNO12bH9oNLQ5wz2KFRZA2hYoyIXEPjUfs6lTLLkuw7rkdAIEujSLqu1QjDHGGFNTjhyGuTe70o+uY2DcC65kojKBQdC0q//jM/WCJddlWJucQYeESMJCanxRSGOMMcbUhtTN8M5ESN0E5zwIQ26rH3XTps6x5LoM6/ZkMKhdbG2HYYwxxpiasP5j+OBGCAqBaz6E9sNrOyJTj1lyXcqBrDz2pOf6vt5aFYoK3a2j6spKA19MnhIY4t/lVY0xxpi6bvE/4d93QYv+cPnrEN2ytiMy9Zwl16UUr8zY3dczhcy9GVI2wvX/qd5tpqUvwrypvotr0jxoW+aELcYYY0zDVlToVihsNwyufg+CGtV2RKYBqDS5FpFmwCNAoqqOFpHuwOmq+rLfo6sFx2YK8WFyrQobP4PsNPfc9YITayc/101en9gP+l5dvZiKCuGz30PSUkuujTHGnJx2LIKs/dD/cUusjc9403P9KjAD+KPn/SbgHaDBJtctYsKICQ/xXaOpm1xiDbDgb9Bl9In1Xn//BhzeC5e+6L5lV9e3T0HKpuq3Y4wxxtRH6z6EoDDofH5tR2IaEG+WP49X1dlAEbiVF4EaWy2xpq1LTvd9SciOhe75jFsheSX89FXV2yjIg4XPQKtBbpUoX0joAqkbfdOWMcYYU58UFcK6udD5PAiJqO1oTAPiTXKdJSJxuOXJEZHBQLpfo6ol2XkFbE3NontzXyfXiyCyGZx1LzRu4Uo7qmr1O5C+C4bd6bupgRK6uJ5rVd+0Z4wxxtQXxSUhPS6p7UhMA+NNcn0HMBfoICILgdeBW/waVS1ZvycTVT/UW29f6FZyCmrk5s3cucht81ZhgRtw0bwPdDzHd7EldIG8TMhI9l2bxhhjTH2w9gNXEtLpvNqOxDQwlSbXqroSGA6cAfwf0ENVV/s7sNqwzjNTSI8WPpyG79AOyEyGNp5Bg/1+CRFNYcFfvW9j7ftwcJtve60B4ru455QNvmvTGGOMqeuKCmH9XFdrbSUhxscqTa5F5JfAVUB/oB8wwbOtwVm3J4OY8GASo0N91+iORe65zRnuOTgMzrgFtn4Nu5ZVfn5RkSsjadodupzgLCPlSfAk16k2qNEY41si8oqI7BeRNeXsFxF5VkS2iMhqEelX0zGak9iOhZCVAj3G1XYkpgHypixkYInHmcADwMV+jKnWrE3OoEdiY8SXvcM7FkJYE0jodmzbgF+5bd96UXu94WM36PDM30GAN/9cVRCR4OJIsUGNxhifexUYVcH+0UAnz2MK8EINxGSMs/ZDCA63khDjF96UhdxS4vFroC/gw3nq6ob8wiI27M30z2DG1mccnxg3ioTBN8Gmf8OeH8o/V9VN3RfX0T8DLkRcaYgl18YYH1PVBcCBCg4ZC7yuzmIgRkSa10x0pkEoyHN3d6uquCSkk80SYvzjRLpCs3E9DQ3KTymHySso8u2y5xl74MDWYyUhJQ2aAo2i3UDF8mz6HPb+6Om1DvRdXCXZdHzGmNrRAthV4n2SZ5sxlTuSCc+cCv+9v+rnHi0JsVlCjH94U3P9sYjM9Tw+ATYCH3nTuIiMEpGNnpq6u8vY/7SIrPI8NonIoRL7Ckvsm1uVD3Ui1u72w8qMxfNbl5Vch0a7BHvdXNhfxoDC4l7rmNbQ6xe+i6m0hC5ugZusVP9dwxhjfq6s+rsy5wUVkSkislxElqekpPg5LFMvLPmXmyxg8QuuE6sq1n5gJSHGr7zpuX4CeNLzeBQYpqo/S5RLE5FA4HlcXV133EDI7iWPUdXfqmofVe0D/B14v8TunOJ9qur3Gu91ezIIDQ6gfUKk7xrdsQhCIuGU3mXvH3Sj+x+8rN7rrV/D7uUw9LcQGOy7mEo7OmOI9V4bY2pUEtCqxPuWQJnzgqrqdFUdoKoDEhISaiQ4U4flpsOiv0Pr093fxy8f9v7cwgJY/7FnlpBw/8VoTmre1Fx/U+KxUFWTvGz7NGCLqm5V1TxgFq7GrjwTgLe9bNvn1ian0+WUxgQG+HIw4yK3omJgOavMR8TBwF/Bmvcg7afj9y34G0Q1hz5X+y6eshydMcSSa2PMz4lIZxH5snjWDxHpLSL3+qDpucAvPbOGDAbSVXWPD9o1Dd2Sf0HuIRj1KJx+s5uuNmmFd+cWl4R0t1lCjP+Um1yLSKaIZJTxyBSRDC/a9rqeTkTaAO2AkuuCh3puAy4WEb/+X6CqrPPMFOIzWWmQsr7skpCSTr8FAkPgu6ePbdu+0P0CGHKbW3jGn6JbQnCE9VwbY8rzInAPkA/gWefgyspOEpG3gf8BXUQkSUSuF5EbROQGzyHzgK3AFs81fuOP4E0Dk3MIFj0HXS6ExL4w5FY389V//uTdasPrbJYQ43/ldKmCqkZVs22v6+lwv6jfU9XCEttaq2qyiLQHvhKRH1X1uO5dEZmCm8KJ1q1bn3CgSQdzyMgt8G1yvfN/7rl48ZjyRDWDftfC8pdh+F0Q08pN0ReR4Lb7mwgkdLbk2hhTnnBVXVpqitKCyk5S1QmV7FfgpmrGZk42i1+AI+kwwlOd2ijKvf70d24Gri6jyz+3sMCNc7KSEONnXs8WIiJNRaR18cOLU7yup8Ml18eVhKhqsud5K/A1bgpASh3jkzq8tcUrM/pyppAdiyAoFFp4sS7CkFsBgYXPuFtbP33lbnXV1P/8Nh2fMaZ8qSLSAU/niIhcBlj5hql5OQdh8T+g20XQvMRYpn7Xuilr/3O/S6DLs2MhZKfaLCHG77yZLeRiEdkMbAO+AbYDn3nR9jKgk4i0E5EQXAL9s1k/RKQL0AR3+7B4WxMRaeR5HQ8MAdZ5cc0Tsi45gwCBLs2q21lfwo6F0HKgd2Ud0S2hz1Ww8nX44o8QGgMDr/ddLJVJ6OxGXed6U+1jjDnJ3AT8C+gqIruB24Ebazckc1L63/NwJANG3HP89sBgOOcBN3Zo1Zvln188S0jHc/0ZpTHll4WU8DAwGPivqvYVkZG4wYcVUtUCEbkZ+BwIBF5R1bUi8hCwXFWLE+0JwCzPLcJi3YB/iUgR7gvAY6rq2+S6MB+2fgOAbNnIFU2OELajjERYgJanQWgVSkZyM2Dvahh2p/fnDP0tfP+mKycZ8Qd3q6umJHR1z6mboWX/mruuMabO89w9PEdEIoAAVc2s7ZjMSSj7gCsJ6T4OmvX4+f6uY6DVYJj/iJu+tvTiMEdnCRllJSHG77xJrvNVNU1EAkQkQFXni8jj3jSuqvNwg1ZKbruv1PsHyjhvEdDLm2ucsLzDMPNSAH5bvG1mOcf2GA+/mOF927uWghZVPpixpNh2cOoEt9z5oCnen+cLR6fj22DJtTHmOCJyX6n3AKjqQ7USkDk5LXoW8rKO1VqXJgLnPQwvn+sGPI646/j9R0tCbJYQ43/eJNeHRCQSWADMFJH9eDGYpc4LiYLr/0t6Th6TZixj0hltGdunjMlMfngLls+AkX+AeC8XptzxHQQEubKQqrjwSTj7PghrUrXzqqtJWzdjiU3HZ4z5uawSr0OBMcD6WorFnIyyUmHJdOg5Hpp2K/+4VqdBt4vd+KUB10Fk02P71n7gZsaykhBTA7xJrscCubgO3quBaKD+91gEBkGrgfywKYXv9RBTuw2CVvE/Py62Hax62y30csk/vWt7xyI3RVDp21KVCQ51j5oWGOQGg9igRmNMKap63CpXIvIEZYyfMcZvFj4DBTkwvNL161zt9cZ58PVjMOYpt80WjjE1rKJ5rp8TkTNUNUtVC1W1QFVfU9VnVTWtJoP0p3V7Kln2PCIeBvwKVs+GA9sqbzAvG3avrFpJSF0Qb9PxGWO8Eg60r+0gzEni8H5Y9hL0vMwNvq9MXAf3N3vFq24cEbi7yVYSYmpQRbOFbAaeFJHtIvK4iPSpqaBq0trkDFrEhBETHlL+QWfcAgGBsHBa5Q3uXg5F+ZXPb13XJHSFQzsgP6e2IzHG1CEi8qOIrPY81gIbgWdqOy5zklj4DBTkunUgvDXs925WkP8+4N6v/dBKQkyNKje5VtVnVPV0YDhwAJghIutF5D4R8eLrY/2wNjmd7pUtHtO4OfS9Br6fCem7Kz52xyJAoPVgn8VYIxI6u0GYaVtqOxJjTN0yBrjI8zgPSFTV52o3JHNSyNzreq17XwnxHb0/LzIBht4GGz6B7d/Bels4xtSsSue5VtUdqvq4qvYFrgIuoYEMZsk6UsC21Cy6N/dimr2htwPqvkVXZMdCOKUXhPpwQZqaUDwdn5WGGGMAEYkVkVggs8QjB2js2W6Mf303zU2bO7wK09oWG3wTRDWHdydBdpotHGNqlDeLyASLyEUiMhO3eMwm4FK/R1YDNuzNRLWCeuuSYlrDqVfCytcgc1/ZxxTkwa5l9a8kBNyARgmw5NoYU2wFsNzzXPqxvBbjMieDjGRY/gr0mQCxJ1DiHxLuZvnKSnElIZ2sJMTUnHJnCxGRc3ELvFwILAVmAVNUNau8c+qbdcXLnrfwspd56B2w6i3433NuPs3S9qxyI5rr22BGcCtJNmlr0/EZYwBQ1Xa1HYM5iX33NGhh1RZjK63P1S5BP6UXBIf5LjZjKlHRVHx/AN4CpqrqgRqKp0atTc4gJjyYxGgvp7+L6wA9L4VlL7sVFcNL3RndsdA918fkGlxpSMqm2o7CGFPHiEgToBNunmsAVHVB7UVkGrSfvnKzffS52nX6nKiAQJj8pbsra0wNqmhA40hVfbGhJtbgkuseiY2PrjjmlTOnQn6WW4a1tO0L3WqHEWXMl10fxHd2AxoL6/8aQcYY3xCRybhFxD4HHvQ8P1CbMZkGbMM8eOsK9/fo7Pur315AoFu90ZgadNJ+ncsvLGLj3kzvBjOW1LSrWwFqyb8gN/3Y9qJC2Lm4/vZag+u5LsqHg17M522MOVncBgwEdqjqSKAvkFK7IZkG6cf34J2Jrozj2o8hIq62IzLmhJy0yfWh7Hz6t2lC/zYnsNT4sKlwJB2WTj+2be+PkJdZPwczFiueoD9lQ+3GYYypS3JVNRdARBqp6gagSy3HZBqalW/AnMluGttffvTzsktj6hFvZgu52VNv16AkRDXi7SmDGdWzedVPbn4qdDof/vcPOHLYbduxyD3X557r+OLk2gY1GmOOShKRGOBD4D8i8hGQXMsxmYZkyb9g7s3Q4Sy4+j1oFFXbERlTLd70XJ8CLBOR2SIySqpUoNyADbsTcg7Aihnu/Y6FbuBFdItaDataGkVB45aQaoMajTGOql6iqodU9QHgT8DLgK0jbXzj2yfhs99D1zEw4W1b6MU0CN4sInMvbpT4y8AkYLOIPCIiHfwcW93WaiC0HwELn4W8bNdzXZ9LQooldLayEGMMIvKpiFwtIhHF21T1G1Wdq6r/z96dx0dZXY8f/5zsO0lIWEMSwLDLGkEWUbQouLAoLihVtK5fqVWrLVZr1Wpr1Z/a1hUVXKpg1aq4oKKiIDsooIQdEghhSdhCgIQs9/fHnQmTkGUSMpmZ5Lxfr3lN5pnneeYwJE9O7px77nFvxqZ81J618PHt9vdizio7F6k6xsA3j9jb6ZfD5a/blrBKNQE1teIrZ4wxIrIb2A2UAHHA+yIy1xjzB08G6NOG3wuvXwRfPWBHsf25JMQpsZttgVRWBgHNtiRfKQXTgKuAZ0XkW2Am8Lkm1qpKezLgjUtsqWRpkd0WFgupw6Dj2dDxLPv7RcQm1l/cB0tfhP7XwsXP2q4eSjURtSbXInIHcB2QB7wK3GuMKRaRAGAT0HyT65ShkDwYVrzmeNwEkuuELlB8FA7tgLgUb0ejlPISY8zHwMciEg6Mwf4eeElEPgdmGmPmejVA5Tv2rreJdWAI/N9iCI6AzAWw7XvYNh/Wf2r3i2xlk+yyEsj4GAbdBqP+rq3yVJPjzsh1AnCpMSbLdaMxpkxELq7pQBEZBfwTCAReNcY8Xun5Z4ARjocRQCtjTKzjueuABxzPPWqMecONWBuXiB29/s+lEN0W4prAgmaJjiYAeRvdT643zYWEtFNr9g+wdx0U5kPyoFM7j1KqwRhjjgHvAu+KSG/gDWyirUONyi489sYlduT5uk/sYmsAva+wN4ADmbBtgU20t82Hgt12zYhzH9DEWjVJ7iTXnwPlC8mISDTQwxiz1BizrrqDRCQQeB4YCWRjJ0XONsZkOPcxxtzlsv9veW2vPQAAIABJREFUsf1TEZF44C9AOmCAlY5jD9TlH9coOp8LnUac+LjL3yV2s/e5GyBtZO3779sC71wBPS+FCa+d2mt/9nt7vt+vbxrvpVJNgIi0Bq7Aloi0Bd4DrvdqUMo35G22iTXAdZ/aQZaqxKXaW/9f25KQosMQVsc1JpTyI+4k1y8C/V0eH6liW1UGApuNMVsBRGQWMBbIqGb/idiEGuACYK5zdUgRmQuMwtb8+RYRuPYjb0fRcCLiISLB/UmNPzwNpsxO6DSm/klxcSFkL4fS47B/64nRD6WUV4jITdjrclfgf8AfjDELvRuV8hn7tsAbF9sSj8mfnlgnoTYimlirJs+dGWtijDHOB8aYMtxLytsDO1weZzu2nfwCIilAR+DbuhwrIjeLyAoRWZGbqwuGNZjEbu614zu4HVbPgqjWcDjHfvRXXztX2sQaTvQMV0p50xDgcaCDMea39UmsHe1bN4jIZhGZWsXzySIyT0R+EpE1InJhQwSuPGz/NjtiXXocrpsNrbp7OyKlfIo7yfVWEblDRIIdt98BW904rqohTFPFNrAfN75vjHH27XHrWGPMNGNMujEmPTEx0Y2QlFuc7fhMdf9dDgv/CQiMfd4+PpWk2HlsaAtNrpXyAcaY640xXzkGVOrMpTRwNNADmCgiPSrt9gDwX2NMP+zvgRdOJWbVCA5k2cS6+KhdSbF1T29HpJTPcSe5vhU7grETO4I8CLjZjeOygQ4uj5OoflWvq6hY8lGXY1VDS+gKhYegYG/1++TvssvV9r0aOp8H4XGnmFwvhFY9odNw+7VSyt+VlwY62vc5SwNdGcBZI9ACvc57x4Yv4KP/g3l/g1XvQNZie42vPMBycIctBSnKt4l1m9O9E69SPq7W8g5jzF5s8ltXy4E0EemITcyvAq6uvJOIdMX2zV7ssvlL4G8uy66fD9xXjxhUfTg7huSuh+jWVe+z6N+21m7YXbYfdvKQ+ifFpcWwY5lN1Ft2hnWfwKFsaJFUv/MppXxBVeV9lVsBPQR85ZjQHgn8qqoTicjNOAZ1kpOTGzzQZm31u/DRrRASDccP2zk0TkHhtmtUXEc7IXHjHDh2CK77GNr28VrISvk6d/pchwG/AXoCYc7txpgbajrOGFMiIlOwiXIgMN0Ys1ZEHgFWGGNmO3adCMyqVNe9X0T+ik3QAR5xTm5UjcC1HV+ns09+/kgerJhu2yzFO9oPpgyBDZ9Bfg7EtKvb6+1aA8VH7DmcExmzFkPvy+v/b1BKNQjHarzZxpgiETkH6A28aYw5WNuhVWyrXGs2EXjdGPP/RGQw8JaI9KpcimKMmYZd1Ib09PRa6tWU2356266o2PEsmDgLAoLtXJoDmXBgm73f77jf9r3tX/3rD6FdPy8HrpRvc2di4lvAemwHj0eAa4BqW/C5MsZ8jm3l57rtwUqPH6rm2OnAdHdeRzWw6LYQGmPb8VVl8fNQUgjD7j6xzbmATtYiOH1C3V7POeKdMhQiExx11ws1uVbKN3wApIvIacBrwGzgHaC2yYfulPf9BtsJCmPMYsdgTgJQQ02aahArX4dPfmfbyV71DgSH2+0Jp9lbZcbYUW1dSVGpWrlTc32aMebPwBHHQi4XAVpo1ZSJ2JUaq2rHd+wALHsFeo6r2HqpTW/7sWJ96q6zFkLL02wJSkAgJJ+pdddK+Y4yY0wJMB541rE+QVs3jisvDRSREGxp4OxK+2wHzgMQke7YT0e19ZOnLXvFJtZp58NVM08k1jUR0cRaKTe5k1wXO+4Pikgv7KSTVI9FpHxDYteq2/EtfdnW5Z11T8XtgUF2ZcW6JtdlpbYExHXp+JQh9rUL9HesUj6gWEQmYldldKxjTXBtBzkScmdp4DpsV5C1IvKIiIxx7PZ74CYRWY2d1D7ZtUSwQRTshc1fN+gp/dqSF+Hze6DrhXDlfyA4rPZjlFJ14k5yPc0xsfAB7KhDBvAPj0alvC+xKxTssSPVToX59sLc9SJo0+vkY1KGQO46OLLP/dfZmwFFh2xJSPl5HF9v15Z8SvmA64HBwGPGmG2OSer/cedAY8znxpguxpjOxpjHHNsedM65McZkGGOGGmP6GGP6GmO+avDo5/wR3v015G1q8FP7nYX/gi+mQvdL4PI3ICjU2xEp1STVmFyLSACQb4w5YIyZb4zpZIxpZYx5uZHiU96S4OwY4jJ6veI1KDwIw39f9TH1SYqdI92uI9dt+9iJM9rvWimvcyTAdxhjZjoGWqKNMY97Oy63XfA3CAqD96+HkiJvR+M985+CuX+GnuNhwgwICvF2REo1WTUm144Z21MaKRblS8o7hjgmNR4/Couesz2t2w+o+ph2/ewvsbokxVkLoUUHiHVprxUUAklnaN21Uj5ARL4TkRgRiQdWAzNE5Glvx+WuQ0EJbB32FOz+Gb5+yNvhND5j4LvH4du/wumXw6WvQmCtVT1KqVPgTlnIXBG5R0Q6iEi88+bxyJR3xSbbRNnZMeTHN+BoHgy/t/pjgkLrlhQbYxNx11Frp5ShsPsXOFZbty+llIe1MMbkA5cCM4wxA6imH7Uv+svsX7j82xjMwFtgyQuw8Utvh+R5pSW2xenyV+HdSfDd36HP1TD+ZTs/RinlUe78lDn7Wd/uss0AnRo+HOUzAgKhZZpNrosL7VLnKcMgZXDNx6UMgflP2hUew1rUvO++zXAkt5rkeghgYMdS6HJBvf8ZSqlTFiQibYErgPu9HUxdDe+SyEercsjo+Xt6Zi2Cj26DWxdCjDsNT/zEsQOQvcJeL3csg50r4XiBfS6yFQz9HZz3kF3wSynlce6s0NixMQJRPiixq71Qr3obDu+C8S/VfkzKENsLdccySBtZ877l/a2HnfxcUrpd0CBroSbXSnnXI9iOHwuNMctFpBPgN7MDz0pLBOC7rYfpOWE6TDsbPrwZfv2Rf7eWMwZ+eAbWvHuibaoEQOte0GcidBhob7Epto2eUqrRuLNC47VVbTfGvNnw4SifktgVfnkfFvw/W+7RsYrVGitLOgMCgmxSXFtynbnQjqo4V2V0FRxua7szte5aKW8yxrwHvOfyeCtwmfciqpvE6FB6tI3h+4253D5iMIx+AmZPsZ/GnXV37SfwVctegW8etiV0Ix6wiXT7ARAa5e3IlGr23CkLOcPl6zBsw/8fAU2umzrnpMb8nXDxM+6NfoREQrv+tU9qNMYm4ClDqj9vyhBY9C8oKtBfGEp5iYgkAf8GhmJLAn8AfmeMyfZqYHUwvEsiry7YyuHCYqL7TYIt38K3j0LqWdDhjNpP4Gu2fGtb6nUZbVdX1HIPpXxKrT+RxpjfutxuAvoB2sOnOXC242vT267k5a6UIbDzR9thpDoHt9uk3bW/dWWpQ6GsBLKXu//a7jq8Bz7/g+3dfSrKSmHuX2DP2oaJqzlb8pJ+UuGbZmDXOGgHtAc+cWzzG8O7JFBSZli8ZZ/9Y/7iZ6BFe/jgBjs/xJ/kbYL/TobEbnDZK5pYK+WD6vNTeRRIa+hAlA9qeRp0u9j2ia1LzV7KUCgrhp0rqt+nqv7WlXUYZGsIPdHv+vt/wLKX7YqTp2LdbFj4rB1FUvVXdBi+/JOtIVW+JtEYM8MYU+K4vQ4kejuoukhPiSciJJD5mxyrvobHwmWvwaGd8Mmd9pM0f3DsAMy8ynb8mDgTQqO9HZFSqgq1Jtci8omIzHbcPgU2AB97PjTldYFBcNXb0PGsuh2XPAiQmpPirIW2m0irHtXvExptF5Rp6OQ6Pwd+egskEJY8bxO7+igrswszSCBsmw/blzZsnM3JjqVgSmH7EttGTPmSPBGZJCKBjtskoA7LsHpfSFAAgzu1ZP7GvBMbOwyEEX+Ctf+zk7arU1ZqF9Na856t0/bWp1SlJfDeZDiQZZctj0vxThxKqVq5U3P9lMvXJUCWP9XaKS8IawFtTq+533XWIkgeUvtHmilD7cSdkqKGW6p30b/tL8xLp8EHv4EV022rqrra+AXs+QUuehrmPWZbEE56v2FibG6cf0AdPwx7frYLEilfcQPwHPAMtuZ6EXZJdL8yvEsi36zfS2beEVITIu3GYXfB1u/g83vtJ2WxKbbzxq7VsHuN4/5nKHYpcZv7ICR2h9Mvg14TIL6RGmp9+Scb65jnav7ETynlde6UhWwHlhpjvjfGLAT2iUiqR6NS/i9lKOxYDiXHT37u8G7Yv8W9XxApQ6C0yNZwN4SCXFgxA3pfCadPgE4jbLJdfKxu5zHGJtNxqdD/Ohh8O2yeCzk/NUyczU3WohOrdOqy9z7FGLPdGDPGGJNojGlljBmHXVDGr5zdxVaylJeGgG3Fd+k0u2DWq+fB39vDy2fZbiKr3rFlaf2vhbEv2N7Yd6+HC5+yAwjfPgr/6guvnAdLXrTXNU9ZMd2WsQ2eAv1/7bnXUUo1CHdGrt8DXLOgUsc2P5xirRpNyhBY+qJNNpMHVXzOmTyl1jCZ0SnZsWhN1sLaF7Bxx5LnoaTwRAuus/8AM0bDj2/CoFvcP8+WbyHnR7jkX7Z85oyb7EfG85+ypTTKfcXH7KIXg26BdZ/a74/Bt9d+nPKmu4FnvR1EXaQmRJIcH8H8jblcOzj1xBMx7WDCdLt6Y6vudgJ3274Q36nqT9YG3mRvB7fDL/+z7Uq/mGpHllOHQfcxEBgCRfm25KzQcV906MTj4mO2S0mvCfaYmvptb5tvR9ZPGwkjH2nw90Up1fDcSa6DjDHlw4/GmOMi4la3EBEZBfwTCAReNcY8XsU+VwAPYT9uXG2MudqxvRT42bHbdmPMGHdeU/kI56h01sKqk+vgSGjTp/bzRMTbuuyshcA9pxbT0f22xKTneEhIOxFnylD44VkYMNm90hPnqHVMe7tYA0BYDAy61U6U3LMWWvc8tVibk50rofS4/X84egA2fG7r2bULgi/zy1VJhndJ4H8/7uR4SRkhQS7fX51H2FtdxCbDsDvtLXcD/Py+TbQ/r3SdCo2x80ec9xHxdp7Gzx/YP+qj2thr0ukTbJ9q18nj+7fCf6+F+M4w4TX/XvRGqWbEneQ6V0TGGGNmA4jIWCCvlmMQkUDgeWAkkA0sF5HZxpgMl33SgPuAocaYAyLSyuUUx4wxfevwb1G+JDLBtorKWnTyQg3OhDvQnW8/bAK8aqad0OPuMVVZ+rJdEnh4pV9+w++Bt8bbj4HT3SglzVoI2xfD6CchyOXvzEG3wuLn7aI7E6bXP87mJmsRIJB8pu2GsOo/kLfBjiIqX+Un7TUqGp6WyH+WbGdF1n6GdE5ouBMndoVz77cTJA9kQmCwTaRDoqv/I/H4Udj0pU3KV7xmP+mLS4VejlruFu3hnavsvlfPsqUoSim/4M7Q0K3An0Rku4hsB/4IuPP5+UBgszFmq2PkexYwttI+NwHPG2MOABhj9rofuvJ5KUNs94ey0hPbju6HvRl1m5CTMgSKj8Du1fWPpTDf/vLqdvHJo8qdRtgRox+ehtLi2s81/0m7smTl2seIeDjjRvtRcZ7frA7tfVkL7ZLN4XEVP/FQXiUih0Ukv4rbYWzPa78zuHNLggKkYteQhiRiJzi2SLLJcE2fvoRE2BHrq96Gezfbuu74TvZTtBcHw7On27kpV7xptyul/IY7i8hsMcacCfQAehpjhhhjNrtx7vbADpfH2Y5trroAXURkoYgscZSROIWJyArH9nFuvJ7yNSlDbfeH3T+f2LZ98Ynn3JXsTLhOYaLb8lftYhFn/f7k50Rg+B9sDeXPtXT72LHcztgfeoddor2ywVPs5Cjt1+ye0mLYsexEUh2bYsttdFKj1xljoo0xMVXcoo0xp/ARkvdEhwXTPyWO+Rtza9+5MYW1gH7XwK8/hN9vsJMm2/WznUE6Dvd2dEqpOnKnz/XfRCTWGFNgjDksInEi8qgb566qJq/yR4lB2AVpzgEmAq+KSKzjuWRjTDpwNfCsiHSuIrabHQn4itxcH7tYKpfJiC6JUtYiCAy1S6S7K6atHbmpb8J1/Agsfg5O+xW0r+Z1u1wArU+3JR2uI+2VLXgKwuNhQDXlI1GJtnZ79Sz78bCqWc4q2+bMmVyL2K8zF/rPwh7Kr5zdJZGMXfnkHi7ydihVi0q0Eyav/Rj6TvR2NEqpenCnLGS0Meag84GjhONCN47LBjq4PE4CcqrY52NjTLExZht2gZo0x+vkOO63At9hl12vwBgzzRiTboxJT0z0qwXDmocW7W0NoetH/FkLISkdgsPqdq6UoTa5LiurexwrX4ej+2D4vdXvI2Jrr/dtgoyPqt5n12rb23rw/0FoVPXnGnqHnXj0g181U/AO5/eGa5lQyhAo2G0ncynVwIan2d8VCzbpgIxSyjPcSa4DRaS8hYKIhAPurOaxHEgTkY6O7iJXAbMr7fMRMMJx3gRsmchWx+h4qMv2oUAGyv84k2JjbBuqXavrtwBCylAoPAi56+p2XHEhLPwXpJ5lJ8zVpPsYSOhq2+lVlcTPfwpCW8DAm2s+T0w76DfJrvp2aGfd4m1ushZByzSIcpnL7CwZ0tIQ5QE928XQMjLE90pDlFJNhjvJ9X+Ab0TkNyJyAzAXeLO2g4wxJcAU4EtgHfBfY8xaEXlERJxt9b7ELkqTAcwD7jXG7AO6AytEZLVj++OuXUaUH0kZAsf221ZVO5aCKatbvbXreaDuCdeq/9hR0JpGrZ0CAmxN9t4M2Din4nN718G62TDoZvdm7Q+905aXLPp33eJtTsocy51X/mMroQtEtNTkWnlEQIAwLC2BBZvyKCvT0iOlVMNzZ0LjE8Cj2IS3J/BXY8w/3Dm5MeZzY0wXY0xnY8xjjm0POtv6GetuY0wPY8zpxphZju2LHI/7OO5fq+8/UHlZ+SjkDzZZCgiCDgPrfp7YZIhJgswf3D+m5LgtzUga6P6koF6XQVxH2xHEteZ3wdO2N/eZ/+feeeJSoM9VsHIGFGgTnCrtWWsX1qj8x5az7lo7higPGZ6WyL4jx8nYle/tUJRSTZBbqzQYY74wxtxjjPk9UCAiz3s4LtVUxKVCdDubWGcutCufhUTW/TzlCdci9ye6rXkXDu2wo9bi5poXgUG2L3fOT7D5G7tt3xa7OMQZv7Ht9tw17G67OMri59w/pjmpaaXOlKFwMAsOZTduTKpZOKuL7XH9vZaGKKU8wK3kWkT6isg/RCQTO4q93qNRqabDmRRvm29X4qtPvbVTyhA4stcmu7UpLbF9q9v2gbSRdXud3lfZUfL5T9hE/oen7XLGg6fU7TwJp0HPS2H5a7a/t6ooa6H9RKJF0snPlZcBLW7cmFSDEpFRIrJBRDaLyNRq9rlCRDJEZK2IvNMYcbWKDqNH2xhNrpVSHlFtci0iXUTkQRFZBzyH7ewhxpgRxhgtJFXuSxkCR3KhrLh+9dbl53GWmLhRLrD2Q9ttoi6j1k5BIXZJ4x1LYfVM21av/3UQ3bruMZ/1e7sq5NKX6n5sU2aMHbmu7vuhdS87eVRLQ/yWyyq9o7HrJEwUkR6V9nFdpbcncGdjxTe8SyI/Zh3gcKEbC0cppVQd1DRyvR44D7jEGDPMkVDX0ABYqWqUJ1Bilz2vr4Q0iEiofaJbWZntR92qB3S9qH6v1W8SRLWGj28HxLbXq4/WPeyqkEtesovYKCtvIxzNq/6TjIBA291Fk2t/5tOr9A7vkkBJmWHxln2N9ZJKqWaiplW2LsO2z5snIl9gL4x1HAJUCkjsars/RLezS1zXl7PEZO3/YMeS6vcrK7W11pe9VvPywzUJDochd8BX90P/a6suXXDX8Hth/afw3BlVr+roasBkGHZX/V8LYN0nsOa/cPnrNkn1ReX9rWv4JCNlCGz6Egpy7cIayt9UtUpv5b+uuwCIyEIgEHjIGPNF5ROJyM3AzQDJyckNElx6SjwRIYHM35TL+T3bNMg5lVIKakiujTEfAh+KSCQwDrgLaC0iLwIfGmO+aqQYlb8TgdFPuNfCrjZDfwfBEZy82GclUa2g5/hTe630G2yNt7sdQqrTri+M/Cvs+aXm/bIW2+XXTzW5XvuhbRuY8ZHtfuKLshbZTwbiO1W/jzPx3r4IelQe8FR+oK6r9CYBC0Skl+vCZWAXDAOmAaSnpzdI/7yQoAAGd2rJ/I15DXE6pZQqV9PINQDGmCPA28DbIhIPXA5MBTS5Vu47fULDnCcp3d4aQ0gEjHykYc7lTlnJVw/A0ml25P1URpxzN9r7+U9Bj/H1H733FGNs55iUITXXw7ftY/+QytLk2k+5u0rvEmNMMbBNRJyr9C5vjACHd0nkm/V7ycw7QmpCPboYKaVUFer0W9cYs98Y87Ix5lxPBaRUs5XQFUqL4EBm/c9RVmrrmWNTql4MxxcczILDObVPbg0KgaQztO7af9V7ld7GCnB4F1tuNF+XQldKNSAfG9JSqhlL7Gbv8zbW/xwHs2yCPuwu22P8+yfc7wveWJwTUt3pHJMyFHb/AscO1r6v8imnuEpvo0htGUGH+HBdCl0p1aA0uVbKVyR2sfe5p9BGPneDvW/d0y5is2vVicVwfEXWQjux1fnHRE1ShgDGtkVUfqe+q/Q2FhFheFoii7bs43hJWWO+tFKqCdPkWilfEdYCotueqJmuD2dyndAF+kysuBiOr8haBMlD3KsFT0qHgGAtDVEec3aXRI4eL2VFli70pJRqGJpcK+VLErpA3ob6H5+7AaLaQHhsxcVwMn9ouBhPRf4uu7iPuyt1BodD+wF2AqRSHjC4c0uCAkS7hiilGowm10r5ksSuduS6viPNeRtOlJfAicVw5j/ZMPGdqvL+1m4m1859d62CogLPxKSateiwYPqnxGndtVKqwWhyrZQvSewKxw9DfuWOZW4wxibmrrXMweEw5Lew7XvYsazh4qyvrEUQEgVtert/TMpQKCuB7EbpzqaaoZHdW5OxK59n5m7E+FIJlVLKL2lyrZQvSehq7+szqTE/xybmCV0qbh9wPYTH+8boddYi6DAIAmttsX9C8iCQgNqXvVeqnq4fmspl/ZP45zeb+MvstZSVaYKtlKo/Ta6V8iWn0o7PWatduQtHaBQMvh02fQU5q04tvlNxZB/krqtbSQhAaLRdUEaTa+UhQYEBPDmhNzed1ZE3F2fxu3dXafcQpVS9aXKtlC+JTLBt6nLrManReUxi15OfG3gThLaABU+dWnynYvtie586rO7Hpgy1ZSElRQ0bk1IOAQHC/Rf1YOrobnyyOocb31zB0eMl3g5LKeWHPJpci8goEdkgIptFZGo1+1whIhkislZE3nHZfp2IbHLcrvNknEr5DBFbGlLf5DosFiITT34urAUMugXWfQJ71516nPWRtQiCwqBdv7ofmzLELo6z88eGj0spF7ee3Zl/XHY6P2zK5ZpXl3Lw6HFvh6SU8jMeS65FJBB4HhgN9AAmikiPSvukAfcBQ40xPYE7Hdvjgb8Ag4CBwF9EJM5TsSrlUxK71q8dX55jMqNI1c+feRsER8J8L41eZy20y5kHhdb92OTBJ86hlIddeUYyL1wzgLU787n8pcXsPlTo7ZCUUn7EkyPXA4HNxpitxpjjwCxgbKV9bgKeN8YcADDG7HVsvwCYa4zZ73huLjDKg7Eq5TsSu8LRfXCkjn13c9dXbMNXWUQ8nPEbWPs/yNt8ajHWVeEh2L2m7vXWThHx0KqHJteq0Yzq1YbXbziDXYcKuezFRWzN1VaQSin3eDK5bg/scHmc7djmqgvQRUQWisgSERlVh2MRkZtFZIWIrMjN1R6lqolw1kzXpTTkSJ5NyGtbUnzIbyEwBH54pv7x1ceOZWDK6p9cgz12+1Io1TpY1TiGdE5g1s1nUlhcyuUvLebn7EPeDkkp5Qc8mVxX9dl05f5GQUAacA4wEXhVRGLdPBZjzDRjTLoxJj0xsYo6U6X8kbMdX11KQ8qXPa9iMqOrqFYwYDKsmQUHsuoVXr1kLYSAIFsWUl8pQ6D4COxe3XBxKVWLXu1b8N6tgwkLDuTKaYt56ssN7D+iddhKqerVodlsnWUDHVweJwGVV8bIBpYYY4qBbSKyAZtsZ2MTbtdjv/NYpEr5khZJtja6LiPX5W34aigLcRpyByx/DX54Gkb+tX4x1lXmD3YiY0hk/c+R7Bj13jIPWqY1TFzuCImCAG2s1Jx1Sozig9uG8Mina3n+u81MX7iNSWemcONZHWkVHebt8JRSPsaTyfVyIE1EOgI7gauAqyvt8xF2xPp1EUnAlolsBbYAf3OZxHg+duKjUk2fiE2S65Jc5260CXlMUu37tmgP/a6Bla/bW2MZ+rtTOz6mLcR3hm//am+N5fcbILpN472e8kltWoTxwjUD2LTnMM/P28yrC7byxqJMJg5M5ubhnWgXG+7tEJVSPsJjybUxpkREpgBfAoHAdGPMWhF5BFhhjJnteO58EckASoF7jTH7AETkr9gEHeARY8x+T8WqlM9J6Arb5ru/f+56SEhzf4T1vL9AYne7rHhjCAiEXhNO/TyXvnKiX3ZjCYlq3NdTPi2tdTTPXtWPO3/VhRe+28x/lmTx9tIsJgxI4razTyO5ZYS3Q1RKeZkY0zSWeU1PTzcrVqzwdhhKNYwFT8M3D8PUHRAWU/v+T/eA1LPg0pc9H5uqlYisNMakezuOpsjXrvXZB47y8vdbeXfFDkrLDGP7tOOm4Z3o3taNn1ullN+q6TqvhYRK+SJnx5C8TbXvW5gP+Tvdq7dWSjWopLgI/jquFwv+MILrh6Qy55fdjP7nAq5+ZQnfrNtDWVnTGMBSSrlPk2ulfJGz60fu+tr3dSbgtXUKUUp5TOuYMB64uAeL7zuXP47qxtbcI/zmjRWc9/T3vLk4kyNF2kJSqeZCk2ulfFFcqu1H7U47PmcCXluPa6WUx8VGhHDbOZ1Z8McR/GtiP2LCg3nw47UM/vs3/P3zdew8eMzbISqlPMyT3UKUUvUVGAQtT7NdQGqTt8FhppjDAAAgAElEQVQm4nGpHg9LKeWe4MAAxvRpxyW92/Lj9oNM/2EbryzYyqs/bGNUrzacdVoCaa2jSWsdRUxYsLfDVUo1IE2ulfJViV0hZ1Xt++VutIl4oP44K+VrRIQBKXEMSIkj+8BR3lycxaxl2/lsza7yfdq2CCOtdTRdWkXRxZFwp7WOJipUf6aV8kf6k6uUr0roChkfQ/ExCK6hh27uemjbp/HiUkrVS1JcBH+6sDtTR3Uj+8AxNu45zMa9h9m0p4CNew7z1tZ9FJWUle/ft0Ms4/u156LebUmICvVi5EqputDkWilfldgFTBns2wxtTq96n+JjcDALel/RuLEppeotIEBIbhlBcssIftWjdfn20jLD9v1H2bjnMOt25fPFL7v5y+y1PPJpBmelJTCub3tG9mhNpI5oK+XTmvRPaHFxMdnZ2RQWFno7FOVjwsLCSEpKIjjYh2sdnRMUczdUn1zv22wT8ETtFKJ8j4iMAv6JXUjsVWPM49XsNwF4DzjDGOM7TawbWWCA0DEhko4JkVzQsw13/qoLG3Yf5qNVO5m9Koc7311FeHAg5/dszbi+7RmWlkBwoPYlUMrXNOnkOjs7m+joaFJTUxERb4ejfIQxhn379pGdnU3Hjh29HU71Wp4GEgB5NUxqdC6Rrm34lI8RkUDgeWAkkA0sF5HZxpiMSvtFA3cASxs/St/XtU00fxzVjXvP78qKrAN8tGonn/+8i49X5RAfGcKIrq04IzWO9NQ4OidG6e86pXxAk06uCwsLNbFWJxERWrZsSW5urrdDqVlQKMR1rLnXdd5Gm4C3PK3x4lLKPQOBzcaYrQAiMgsYC2RU2u+vwBPAPY0bnn8JCBAGdoxnYMd4HrqkJ99vzGX26hy+27CXD37MBiAuIpgBKfGkp8ZxRmocvdq3IDQo0MuRK9X8NOnkGtDEWlXJb74vErvW3I4vd71twRcc1mghKeWm9sAOl8fZwCDXHUSkH9DBGPOpiGhy7aaQoABG9mjNyB6tMcawLe8IK7IOsCJzPysyD/D1uj3l+/VJakHvpFg6J0bRKTGSzolRJESF+M81UCk/1OSTa2/at28f5513HgC7d+8mMDCQxMREAJYtW0ZISEit57j++uuZOnUqXbvW7WP/iy66iPz8fBYsWFD3wJXvSOgCm+ZCaUnVrfZyN2pJiPJVVWVv5WuBi0gA8AwwudYTidwM3AyQnJzcQOE1DSJCp8QoOiVGcUV6BwDyCopYmXWAlVkHWLZtP28vzaKw+EQXkuiwIDonRlVIuDslRpIcH0FYsI50K3WqNLn2oJYtW7Jqle1T/NBDDxEVFcU991QcnDHGYIwhIKDqSSkzZsyo8+vu27ePn3/+mbCwMLZv3+6xX0YlJSUEBem3kEcldoOyYjiwDRLSKj5XWmInNHa5wDuxKVWzbKCDy+MkIMflcTTQC/jOMYraBpgtImMqT2o0xkwDpgGkp6cbVI0SokK5oGcbLujZBoCyMsOu/EK27C1gS24BW3OPsCW3gB8255aXlACIQLsW4aQmRJDa0k6sTG0ZSWqCTbxDgnTypFLu0MzICzZv3sy4ceMYNmwYS5cu5dNPP+Xhhx/mxx9/5NixY1x55ZU8+OCDAAwbNoznnnuOXr16kZCQwK233sqcOXOIiIjg448/plWrVied//3332fcuHG0aNGCd999l3vvvRewo+e33HIL27ZtQ0SYNm0agwYNYsaMGTzzzDOICP3792fGjBlMmjSJCRMmMG7cOACioqIoKCjg66+/5vHHHychIYG1a9fy888/c8kll5CTk0NhYSF33XUXN954IwCfffYZf/7znyktLaV169bMmTOHrl27smzZMuLj4yktLSUtLY0VK1YQHx/fSO++n0nsYu9zN5ycXB/YZhNv7RSifNNyIE1EOgI7gauAq51PGmMOAQnOxyLyHXBPc+4W4ikBAUL72HDax4YzvEtihecOFxazLe8IW3OPkLnvCJl5R9i27yifrtnFoWPFJ84h0DEhknO7tWJkjzYMSIkjMEBLS5SqSrNJrh/+ZC0ZOfkNes4e7WL4yyU963VsRkYGM2bM4KWXXgLg8ccfJz4+npKSEkaMGMGECRPo0aNHhWMOHTrE2WefzeOPP87dd9/N9OnTmTp16knnnjlzJn//+99p0aIFkyZNKk+ub7/9dkaOHMmUKVMoKSnh6NGjrF69mn/84x8sWrSI+Ph49u/fX2vsS5YsISMjo3xE/I033iA+Pp6jR4+Snp7OZZddRlFREbfddhsLFiwgJSWF/fv3ExgYyMSJE3nnnXeYMmUKX375JWeccYYm1jVJcCbX66H7xRWfc3YK0eRa+SBjTImITAG+xLbim26MWSsijwArjDGzvRuhAogOC6Z3Uiy9k2JPeu7AkeNscyTcmXlHWJ19iDcWZfHKgm20jAzh3G6tOL9nG85KS9ByEqVcNJvk2td07tyZM844o/zxzJkzee211ygpKSEnJ4eMjIyTkuvw8HBGjx4NwIABA6qsp965cyfbt2/nzDPPREQoLS1l/fr1dOvWje+++45Zs2YBEBQURExMDN9++y1XXnlleYLrTqI7ePDgCqUmzzzzDLNn29+T2dnZbNmyhR07djBixAhSUlIqnPc3v/kNl19+OVOmTGH69Onlo9yqGqHREJNUdTs+ZxcRZwKulI8xxnwOfF5p24PV7HtOY8Sk3BcXGUJcZAj9k+PKtx0uLOb7jbnMzdjDF2t3897KbMKCAxielsjIHq05r3tr4iNrn0+kVFPm0eS6tgUERGQy8CT2I0OA54wxrzqeKwV+dmzfbowZcyqx1HeE2VMiIyPLv960aRP//Oc/WbZsGbGxsUyaNKnKhW9cJ0AGBgZSUlJy0j7vvvsu+/btK+/ffOjQIWbNmsVDDz0EnNwlwxhT5azxoKAgysrsBJjS0tIKr+Ua+9dff838+fNZsmQJ4eHhDBs2jMLCwmrPm5qaSlxcHPPmzeOnn37i/PPPr/L9US4Su1Tdji9vI8S0twm4Uko1guiwYC7u3Y6Le7fjeEkZy7bt56uM3czN2MNXGbZLSYf4cLq1iaF72xi6t4mme9sYkuMjCNAyEtVMeCy5dncBAeBdY8yUKk5xzBjT11Px+ZL8/Hyio6OJiYlh165dfPnll4waNape55o5cyZff/11+aj4pk2buPjii3nooYcYMWIEL730ElOmTKG0tJQjR47wq1/9iiuuuII77rijvCwkPj6e1NRUVq5cyaWXXsqHH35IaWlpla936NAh4uPjCQ8PZ+3atSxfvhyAoUOHcuedd5KVlVVeFuI6en3NNddw/fXXVzuRU7lI7AYrX4eyMnB9v3LXa0mIUsprQoICGJaWwLC0BB4e05NfduYzf1Mu63bls373Yb5Zt4cyx/TTiJBAujoS7W5toukQH0H72HDaxYYTpcu5qybGk9/R7i4g0Oz179+fHj160KtXLzp16sTQoUPrdZ4tW7awe/du0tPTy7elpaURGhrKypUree6557jpppt4+eWXCQoK4uWXX2bgwIH84Q9/YPjw4QQFBTFgwABee+01brnlFsaOHcvcuXM5//zzCQ0NrfI1L7roIqZNm0afPn3o1q0bgwbZNratW7fmxRdfZOzYsRhjaNeuHXPmzAFg/Pjx3HDDDUyePLle/85mJ6ELFB+F/GyIdZTjlJVB3iboP8S7sSmlFPZT0dOTWnB6UovybYXFpWzcc5j1uw6TsSufdbvy+XR1Du8srfipa0xYEO0cEy7bld/C6JQQRedWkUSEaPKt/IsY45muRiIyARhljLnR8fjXwCDXUWpHWcjfgVxgI3CXMWaH47kSYBVQAjxujPmoitdw7X06ICsrq8Lz69ato3v37g3/j1OnZMmSJdx3333MmzfPq3H4zfdH1mKYMQqueR/SRtptB7Lgn73h4mch/XrvxqdOIiIrjTHpte+p6io9Pd2sWKENRfyVMYY9+UXsPHiUnQcLyTl4rPzmfOzapQSgfWw4nVtFcVpiFKe1OnHT2m7lTTVd5z3552CNCwg4fALMNMYUicitwBvAuY7nko0xOSLSCfhWRH42xmypcDLtfep3HnvsMaZNm1Y+sVK5wVn6kbvhRHLtnOCoZSFKKT8iIrRpEUabFmEMSKl6n4KiEnIOHmNrbgGb99rbpr0FLNu2r8JiOPGRIbSMDCE8JJCw4EDCnTfXxyEBtI+NoG+HWLq0jiIoUEsRled5MrmubQEBjDH7XB6+AvzD5bkcx/1WR//TfkCF5Fr5n/vvv5/777/f22H4l4h4iEioOKnR2YZPV2dUSjUxUaFBdGkdTZfWFSdrl5UZdh48xubcgvIFcQ4dK+bY8VKOFZdy8Fgxuw8VcqzYPi48XsrR4lJKHYXf4cGBnN6+BX2TY+nbIZY+HWJp1yJMl4JXDc6TyXWNCwgAiEhbY8wux8MxwDrH9jjgqGNEOwEYCjzhwViV8m2J3Sq248tdbxPuyJbei0kppRpRQIDQIT6CDvERjOh68gJqVTHGkLXvKKuzD/LT9oOszj7I6wszOV5qR8ATo0PpkxRL76QWdGkdXT7ZUhfIUafCY8m1mwsI3CEiY7B11fuByY7DuwMvi0gZEICtudaJkKr5SuwCv/wPjLFrFOdt1JIQpZSqhYiQmmCXcB/btz0Ax0vKWLcrn9XZB1m1/SCrdhzk63V7yo8JCw4oHznv1ubEfWJ0qI5yK7d4dApubQsIGGPuA+6r4rhFwOmejE0pv5LYDQoPQsFeiGplR657XebtqJRSyu+EBAXQx1EWcu1gu+1IUQmb9hawcfdh1u8+zMY9h/luQy7vr8wuPy46NIjWLcJoExNGq5hQ2sTY2vHWMXZb65gwWkaFEBQgmoQ3c9rfRil/4FyFMc9Ra114SOutlVKqgUSGBtG3g63FdrWvoIgNew6zcfdhtuUdYU9+EbvzC9m6pYC9h4soKau6l4IIBIoQIEJAAASI2McBQkhQAO1jw0ltGUFyy0hS4iNITYggOT6ShKgQTcybAE2uPeicc87hvvvu44ILLijf9uyzz7Jx40ZeeOGFao+LioqioKCAnJwc7rjjDt5///0qz/3UU09V6Gld2bPPPsvNN99MREQEABdeeCHvvPMOsbGx1R5TF3369KFHjx7MnDmzQc6nauDaMaR8my57rpRSntQyKpQhUaEM6Zxw0nNlZYa8I0XszS9i96FCducXcuDIcUqNoazM2Htj9yt1PDYGjh0vZceBoyzPPMDs1Tm45ueRIYF0iI+gU2IkI3u05oKebbTPtx/S/zEPmjhxIrNmzaqQXM+aNYsnn3zSrePbtWtXZWLtrmeffZZJkyaVJ9eff/55LUe4b926dZSVlTF//nyOHDlSYUn0hlRSUkJQkH6bEt0WQmMqJdfdvBePUko1cwEBQqvoMFpFh9GrfYvaD6jC8ZIysg8cJWvfUbL2HSFr/1G27zvKqu0H+fzn3USG/MKFp7flsgFJDEyNr/MS8qVlhgBBR8MbmWYtHjRhwgQeeOABioqKCA0NJTMzk5ycHIYNG0ZBQQFjx47lwIEDFBcX8+ijjzJ27NgKx2dmZnLxxRfzyy+/cOzYMa6//noyMjLo3r07x44dK9/vtttuY/ny5Rw7dowJEybw8MMP869//YucnBxGjBhBQkIC8+bNIzU1lRUrVpCQkMDTTz/N9OnTAbjxxhu58847yczMZPTo0QwbNoxFixbRvn17Pv74Y8LDw0/6t73zzjv8+te/Zt26dcyePZuJEycCsHnzZm699VZyc3MJDAzkvffeo3PnzjzxxBO89dZbBAQEMHr0aB5//PEKo+95eXmkp6eTmZnJ66+/zmeffUZhYSFHjhxh9uzZ1b5Xb775Jk899RQiQu/evXnhhRfo3bs3GzduJDg4mPz8fHr37s2mTZsIDg721H+154nY0hBnWUhItE24lVJK+a2QoAA6JUbRKTGqwvayMsPyzP188GM2n/+8m/dWZpMUF86l/ZO4rH97UlqePKB17Hgp63bnszYnn4ycQ6zNscvQC1SoDa/4dSitY8Jo2yJcO6Q0oOaTXM+ZCrt/bthztjkdRj9e7dMtW7Zk4MCBfPHFF4wdO5ZZs2Zx5ZVXIiKEhYXx4YcfEhMTQ15eHmeeeSZjxoyp9q/LF198kYiICNasWcOaNWvo379/+XOPPfYY8fHxlJaWct5557FmzRruuOMOnn76aebNm0dCQsWPs1auXMmMGTNYunQpxhgGDRrE2WefTVxcHJs2bWLmzJm88sorXHHFFXzwwQdMmjTppHjeffdd5s6dy4YNG3juuefKk+trrrmGqVOnMn78eAoLCykrK2POnDl89NFHLF26lIiICPbv31/rW7t48WLWrFlDfHw8JSUlVb5XGRkZPPbYYyxcuJCEhAT2799PdHQ055xzDp999hnjxo1j1qxZXHbZZf6dWDsldoPNc23HkMSuNuFWSinV5AQECIM6tWRQp5Y8PKYXX67dzQc/ZvPvbzfxr282cUZqHGP6tqeouJRfdtpEektuQXmJSWxEMD3bxXDdYLtSz+78IvYcKmTVjoPsXlvI8ZKyCq8XGRJI76RY+ibH0q+DvW8VHdbY/+wmo/kk117iLA1xJtfO0WJjDH/605+YP38+AQEB7Ny5kz179tCmTZsqzzN//nzuuOMOAHr37k3v3r3Ln/vvf//LtGnTKCkpYdeuXWRkZFR4vrIffviB8ePHl5dyXHrppSxYsIAxY8bQsWNH+vbtC8CAAQPIzMw86fjly5eTmJhISkoKSUlJ3HDDDRw4cICgoCB27tzJ+PHjAQgLsz+YX3/9Nddff315eUp8fHyt79vIkSPL96vuvfr222+ZMGFC+R8Pzv1vvPFGnnjiCcaNG8eMGTN45ZVXan09v5DYBVb9B4qPQfdLvB2NUkqpRhAeEsi4fu0Z1689OQeP8eFPO/ngx2z+/NEvALSJCaNnuxhGn96Wnu1i6NW+RY2L4xhjOHi0mN35tk5896FC1u/K56cdB3ll/tbySZrtY8Pp51hwp19yHKclRhEVFqQj3G5oPsl1DSPMnjRu3DjuvvtufvzxR44dO1Y+4vz222+Tm5vLypUrCQ4OJjU1lcLCwhrPVdUPyrZt23jqqadYvnw5cXFxTJ48udbzGFP9SvGhoaHlXwcGBlYoP3GaOXMm69evJzU1FYD8/Hw++OADrrjiimpfr6rYg4KCKCuzfz1Xjtm1hru696q68w4dOpTMzEy+//57SktL6dWrV7X/Xr/irLEuytce10op1Qy1iw3n9hGn8X/ndGZLbgGxESEkRIXWfqALESEuMoS4yBC6t42p8FxhcSlrcw7x0/aD5bdP1+yqsE9kSCBRYUFEhQYRFRZMdKjz6yAiQwIJCQqwt0DXr6X868iQIJLiIugQH050WBP4VLkKzSe59pKoqCjOOeccbrjhhvLSCYBDhw7RqlUrgoODmTdvHllZWTWeZ/jw4bz99tuMGDGCX375hTVr1gA2sY2MjKRFixbs2bOHOXPmcM455wAQHR3N4cOHTyoLGT58OJMnT2bq1KkYY/jwww9566233Pr3lJWV8d5777FmzRrat7cN+efNm8ejjz7KjTfeSFJSEh999BHjxo2jqKiI0tJSzj//fB555BGuvvrq8rKQ+Ph4UlNTWblyJQMHDqxx4mZ179V5553H+PHjueuuu2jZsmX5eQGuvfZaJk6cyJ///Ge3/l1+IcGlO4i24VNKqWZLRDitVXTtO9ZRWHAgA1LiGZBy4hPmPfmF/LT9INkHjnK4sISCohIKHPeHi0ooKCxm7+FCCgpLOHK8lOMlZRwvLStfdr4mcRHB5atudnAk3MnxEbSLDScsOJDgQCEkMICgwACCA4XggIA6T+r0Bk2uG8HEiRO59NJLmTVrVvm2a665hksuuYT09HT69u1Lt241d3647bbbuP766+nduzd9+/Zl4MCBgG2H169fP3r27EmnTp0YOnRo+TE333wzo0ePpm3btsybN698e//+/Zk8eXL5OW688Ub69etXZQlIZfPnz6d9+/bliTXYZD0jI4Ndu3bx1ltvccstt/Dggw8SHBzMe++9x6hRo1i1ahXp6emEhIRw4YUX8re//Y177rmHK664grfeeotzzz232tes7r3q2bMn999/P2effTaBgYH069eP119/vfyYBx54oMIfNH4vNhmCwqCkUNvwKaWUahStY8IY1avqktWalJYZm2iXlFFUWlr+dUFRCdkHjrF9/1F27D/K9v1HycjJ56u1uykurT0hDwwQggOF0KBAUltG0LlVFGmtoklrFUVa6yiS4ry/fL3UVCLgT9LT082KFSsqbFu3bh3du3f3UkTKm95//30+/vjjGkfk/fL746VhkLcJ/pQDAYHejkZVQ0RWGmOqb0Kv6q2qa71Syv+Vlhn25Beyff9Rdh8qLB8BLy4to6TUlH9tb4ajx0vIzDvKpr2H2ZNfVH6eUEcHlrRWUXROjEIEO8pePupeXPFxUQlXnZHM1NF1a29b03VeR65Vk/Pb3/6WOXPmNGhfb5/R8WyIaqOJtVJKqSYlMEBoFxtOu9iT2//WJr+wmM17C9i8p4BNew+zaW8BK7PsIj0AYcEBRIUGExMWVF4vnhwfQVRYENGO1TkbkibXqsn597//7e0QPOeCx7wdgVJKKeVTYsKC6Z8cR//kuArbC4tLHWUkAY0ajybXSimllFKqyQkL9s6nvI2byntBU6kpVw1Lvy+UUkop5QlNOrkOCwtj3759mkipCowx7Nu3r3yRG6WUUkqphtKky0KSkpLIzs4mNzfX26EoHxMWFkZSUpK3w1CqSRORUcA/gUDgVWPM45Wevxu4ESgBcoEbjDE1N/1XSikf59Hk2o0L62TgSWCnY9NzxphXHc9dBzzg2P6oMeaNur5+cHAwHTt2rGf0Siml6ktEAoHngZFANrBcRGYbYzJcdvsJSDfGHBWR24AngCsbP1qllGo4Hkuu3bywArxrjJlS6dh44C9AOmCAlY5jD3gqXqWUUg1qILDZGLMVQERmAWOB8t8Bxph5LvsvASY1aoRKKeUBnqy5Lr+wGmOOA84LqzsuAOYaY/Y7Euq5wCgPxamUUqrhtQd2uDzOdmyrzm+AOVU9ISI3i8gKEVmhZX5KKV/nyeTa3QvrZSKyRkTeF5EOdTxWKaWUb6pq/eEqZ5eLyCTsJ5VPVvW8MWaaMSbdGJOemJjYgCEqpVTD82TNtTsX1k+AmcaYIhG5FXgDONfNYxGRm4GbHQ8LRGRDpV0SgLw6Re19/hgz+Gfc/hgzaNyNqb4xpzR0IH4oG+jg8jgJyKm8k4j8CrgfONsYU1T5+cpWrlyZJyKukx798fsK/DNuf4wZNO7G5I8xQ/3irvY678nkutYLqzFmn8vDV4B/uBx7TqVjv6v8AsaYacC06gIQkRXVrfvuq/wxZvDPuP0xZtC4G5M/xuxDlgNpItIRO2n9KuBq1x1EpB/wMjDKGLPXnZMaYyoMXfvr/5E/xu2PMYPG3Zj8MWZo+Lg9WRZSfmEVkRDshXW26w4i0tbl4RhgnePrL4HzRSROROKA8x3blFJK+QFjTAkwBXvtXgf81xizVkQeEZExjt2eBKKA90RklYjMruZ0SinlNzw2cm2MKRER54U1EJjuvLACK4wxs4E7HBfZEmA/MNlx7H4R+Ss2QQd4xBiz31OxKqWUanjGmM+Bzytte9Dl6181elBKKeVhHu1z7caF9T7gvmqOnQ5MP8UQqi0Z8WH+GDP4Z9z+GDNo3I3JH2Nubvz1/8gf4/bHmEHjbkz+GDM0cNyiS4MrpZRSSinVMDxZc62UUkoppVSz0iSTaxEZJSIbRGSziEz1djzuEpFMEfnZMbFnhbfjqY6ITBeRvSLyi8u2eBGZKyKbHPdx3oyxsmpifkhEdjre71UicqE3Y6yKiHQQkXkisk5E1orI7xzbffb9riFmn36/RSRMRJaJyGpH3A87tncUkaWO9/pdxwRt5QP88Vqv13nP8sdrvT9e58E/r/WNdZ1vcmUhYpdd34jLsuvAxCqWXfc5IpIJpBtjfLpHpIgMBwqAN40xvRzbngD2G2Med/ySizPG/NGbcbqqJuaHgAJjzFPejK0mjo46bY0xP4pINLASGIed/OuT73cNMV+BD7/fIiJApDGmQESCgR+A3wF3A/8zxswSkZeA1caYF70Zq/Lfa71e5z3LH6/1/nidB/+81jfWdb4pjlyfyrLryg3GmPnY7i6uxmIXAcJxP65Rg6pFNTH7PGPMLmPMj46vD2NbmrXHh9/vGmL2acYqcDwMdtwMdmGr9x3bfeq9bub0Wu9B/nidB/+81vvjdR7881rfWNf5pphc+/PS6Qb4SkRWil190p+0NsbsAvsDB7TycjzumiIiaxwfJfrUR26ViUgq0A9Yip+835ViBh9/v0UkUERWAXuBucAW4KCjZzP41/WkqfPXa71e573Dp689Tv54nQf/utY3xnW+KSbXbi2d7qOGGmP6A6OB2x0fbynPeRHoDPQFdgH/z7vhVE9EooAPgDuNMfnejscdVcTs8++3MabUGNMXuyrsQKB7Vbs1blSqGv56rdfrfOPz+WsP+Od1HvzvWt8Y1/mmmFzXuuy6rzLG5Dju9wIfYv/T/cUeR/2Vsw7LraWMvckYs8fxQ1YGvIKPvt+OurAPgLeNMf9zbPbp97uqmP3l/QYwxhwEvgPOBGJFxLkmgN9cT5oBv7zW63W+8fnDtccfr/Pg39d6T17nm2JyXeuy675IRCIdEwIQkUjsku+/1HyUT5kNXOf4+jrgYy/G4hbnRcthPD74fjsmX7wGrDPGPO3ylM++39XF7Ovvt4gkikis4+tw4FfYGsJ5wATHbj71Xjdzfnet1+u8d/jBtcfvrvPgn9f6xrrON7luIQCOti/PcmLZ9ce8HFKtRKQTdhQD7MqZ7/hq3HHXifsAAAK7SURBVCIyEzgHSAD2AH8BPgL+CyQD24HLfWnJ+mpiPgf7sZUBMoFbnPVtvkJEhgELgJ+BMsfmP2Hr2nzy/a4h5on48PstIr2xE1kCsQMP/zXGPOL42ZwFxAM/AZOMMUXei1Q5+du1Xq/znueP13p/vM6Df17rG+s63ySTa6WUUkoppbyhKZaFKKWUUkop5RWaXCullFJKKdVANLlWSimllFKqgWhyrZRSSimlVAPR5FoppZRSSqkGosm1apJEpFREVrncpjbguVNFxGf6diqlVHOk13nlq4Jq30Upv3TMsbypUkqppkmv88on6ci1alZEJFNE/iEiyxy30xzbU0TkGxFZ47hPdmxvLSIfishqx+3/t3fHqlFEURzGvz8hSEBstBEsbFIJChIsLH0FiyhWkipNrMQX8AmCNhEsBN9BFAtBFDubtGKXQFKIpBGRY7FXGCVBkLvOsvv9mjlzZlnuNIczd+/svd6+ainJkyS7SV62nZ4kSSOzzmtsNteaVyt//Fy4Prj2taquAY+Y7O5Gi59V1WXgObDd8tvAm6q6AlwFdlt+FXhcVZeAL8DNKd+PJOl31nnNJHdo1FxKclRVp4/JfwZuVNWnJMvAflWdTXIInK+q7y2/V1XnkhwAF4bboCa5CLyqqtV2/gBYrqqH078zSRJY5zW7nLnWIqoT4pM+c5xvg/gHvr8gSbPEOq/R2FxrEa0Pju9b/A641eI7wNsWvwY2AZIsJTnzvwYpSfpn1nmNxqcwzauVJB8H5y+q6tffNJ1K8oHJw+XtltsCnia5DxwAd1v+HrCTZIPJzMUmsDf10UuS/sY6r5nkmmstlLYWb62qDsceiySpP+u8xuayEEmSJKkTZ64lSZKkTpy5liRJkjqxuZYkSZI6sbmWJEmSOrG5liRJkjqxuZYkSZI6sbmWJEmSOvkJPPzCRFd0v9oAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "#可視化結果\n", - "f, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n", - "t = f.suptitle('Basic CNN Performance', fontsize=12)\n", - "f.subplots_adjust(top=0.85, wspace=0.3)\n", - "\n", - "epoch_list = list(range(1,31))\n", - "ax1.plot(epoch_list, history.history['acc'], label='Train Accuracy')\n", - "ax1.plot(epoch_list, history.history['val_acc'], label='Validation Accuracy')\n", - "ax1.set_xticks(np.arange(0, 31, 5))\n", - "ax1.set_ylabel('Accuracy Value')\n", - "ax1.set_xlabel('Epoch')\n", - "ax1.set_title('Accuracy')\n", - "l1 = ax1.legend(loc=\"best\")\n", - "\n", - "ax2.plot(epoch_list, history.history['loss'], label='Train Loss')\n", - "ax2.plot(epoch_list, history.history['val_loss'], label='Validation Loss')\n", - "ax2.set_xticks(np.arange(0, 31, 5))\n", - "ax2.set_ylabel('Loss Value')\n", - "ax2.set_xlabel('Epoch')\n", - "ax2.set_title('Loss')\n", - "l2 = ax2.legend(loc=\"best\")" - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Model: \"sequential_2\"\n", - "_________________________________________________________________\n", - "Layer (type) Output Shape Param # \n", - "=================================================================\n", - "conv2d_5 (Conv2D) (None, 98, 98, 32) 896 \n", - "_________________________________________________________________\n", - "max_pooling2d_5 (MaxPooling2 (None, 49, 49, 32) 0 \n", - "_________________________________________________________________\n", - "conv2d_6 (Conv2D) (None, 47, 47, 64) 18496 \n", - "_________________________________________________________________\n", - "max_pooling2d_6 (MaxPooling2 (None, 23, 23, 64) 0 \n", - "_________________________________________________________________\n", - "conv2d_7 (Conv2D) (None, 21, 21, 128) 73856 \n", - "_________________________________________________________________\n", - "max_pooling2d_7 (MaxPooling2 (None, 10, 10, 128) 0 \n", - "_________________________________________________________________\n", - "conv2d_8 (Conv2D) (None, 8, 8, 128) 147584 \n", - "_________________________________________________________________\n", - "max_pooling2d_8 (MaxPooling2 (None, 4, 4, 128) 0 \n", - "_________________________________________________________________\n", - "flatten_2 (Flatten) (None, 2048) 0 \n", - "_________________________________________________________________\n", - "dropout_1 (Dropout) (None, 2048) 0 \n", - "_________________________________________________________________\n", - "dense_3 (Dense) (None, 512) 1049088 \n", - "_________________________________________________________________\n", - "dense_4 (Dense) (None, 3) 1539 \n", - "=================================================================\n", - "Total params: 1,291,459\n", - "Trainable params: 1,291,459\n", - "Non-trainable params: 0\n", - "_________________________________________________________________\n" - ] - } - ], - "source": [ - "#套入VGG16模型\n", - "from keras.applications import VGG16\n", - "\n", - "\n", - "conv_base = VGG16(weights='imagenet',\n", - " include_top=False,\n", - " input_shape=(100,100, 3))\n", - "model.summary()" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Found 60 images belonging to 3 classes.\n", - "[[1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [1. 0. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]]\n", - "[[0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 1. 0.]]\n", - "[[0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 1. 0.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [0. 0. 1.]]\n", - "Found 27 images belonging to 3 classes.\n", - "[[0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [1. 0. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]]\n", - "Found 15 images belonging to 3 classes.\n", - "[[0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [1. 0. 0.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]\n", - " [0. 0. 1.]\n", - " [0. 0. 1.]\n", - " [0. 1. 0.]\n", - " [1. 0. 0.]]\n" - ] - } - ], - "source": [ - "#使用conv_base的模型先抽取特徵跟label產出，將特徵提取後進行降維輸出\n", - "import os\n", - "import numpy as np\n", - "from keras.preprocessing.image import ImageDataGenerator\n", - "\n", - "base_dir = 'Rec'\n", - "\n", - "train_dir = os.path.join(base_dir, 'train')\n", - "validation_dir = os.path.join(base_dir, 'validation')\n", - "test_dir = os.path.join(base_dir, 'test')\n", - "\n", - "datagen = ImageDataGenerator(rescale=1./255)\n", - "batch_size = 20\n", - "\n", - "def extract_features(directory, sample_count):\n", - " features = np.zeros(shape=(sample_count, 3, 3, 512))\n", - " labels = np.zeros(shape=(sample_count,3))\n", - " generator = datagen.flow_from_directory(\n", - " directory,\n", - " target_size=(100, 100),\n", - " batch_size=batch_size,\n", - " class_mode='categorical'\n", - " )\n", - " i = 0\n", - " for inputs_batch, labels_batch in generator:\n", - " features_batch = conv_base.predict(inputs_batch)\n", - " features[i * batch_size : (i + 1) * batch_size] = features_batch\n", - " print(labels_batch)\n", - " labels[i * batch_size : (i + 1) * batch_size] = labels_batch\n", - " i += 1\n", - " if i * batch_size >= sample_count:\n", - " \n", - " break\n", - " return features, labels\n", - "\n", - "train_features, train_labels = extract_features(train_dir, 60)\n", - "validation_features, validation_labels = extract_features(validation_dir, 20)\n", - "test_features, test_labels = extract_features(test_dir, 15)" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "metadata": {}, - "outputs": [], - "source": [ - "train_features = np.resize(train_features, (60, 3 * 3 * 512))\n", - "validation_features = np.resize(validation_features, (20, 3 * 3 * 512))\n", - "test_features = np.resize(test_features, (15, 3 * 3 * 512))" - ] - }, - { - "cell_type": "code", - "execution_count": 24, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Train on 60 samples, validate on 20 samples\n", - "Epoch 1/30\n", - "60/60 [==============================] - 1s 10ms/step - loss: 2.0313 - acc: 0.3167 - val_loss: 1.8868 - val_acc: 0.3000\n", - "Epoch 2/30\n", - "60/60 [==============================] - 0s 483us/step - loss: 1.8181 - acc: 0.3333 - val_loss: 1.7381 - val_acc: 0.3000\n", - "Epoch 3/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 1.6853 - acc: 0.3000 - val_loss: 1.6217 - val_acc: 0.3000\n", - "Epoch 4/30\n", - "60/60 [==============================] - 0s 550us/step - loss: 1.6355 - acc: 0.3167 - val_loss: 1.5229 - val_acc: 0.3000\n", - "Epoch 5/30\n", - "60/60 [==============================] - 0s 483us/step - loss: 1.5393 - acc: 0.3333 - val_loss: 1.4473 - val_acc: 0.3000\n", - "Epoch 6/30\n", - "60/60 [==============================] - 0s 550us/step - loss: 1.5782 - acc: 0.2833 - val_loss: 1.3734 - val_acc: 0.3000\n", - "Epoch 7/30\n", - "60/60 [==============================] - 0s 533us/step - loss: 1.4986 - acc: 0.2833 - val_loss: 1.3062 - val_acc: 0.3000\n", - "Epoch 8/30\n", - "60/60 [==============================] - 0s 500us/step - loss: 1.4321 - acc: 0.3667 - val_loss: 1.2484 - val_acc: 0.3000\n", - "Epoch 9/30\n", - "60/60 [==============================] - 0s 583us/step - loss: 1.3253 - acc: 0.3333 - val_loss: 1.1981 - val_acc: 0.3000\n", - "Epoch 10/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 1.2468 - acc: 0.4167 - val_loss: 1.1543 - val_acc: 0.3000\n", - "Epoch 11/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 1.1066 - acc: 0.4500 - val_loss: 1.1124 - val_acc: 0.3000\n", - "Epoch 12/30\n", - "60/60 [==============================] - 0s 600us/step - loss: 1.2571 - acc: 0.3333 - val_loss: 1.0764 - val_acc: 0.3000\n", - "Epoch 13/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 1.1691 - acc: 0.4667 - val_loss: 1.0375 - val_acc: 0.3500\n", - "Epoch 14/30\n", - "60/60 [==============================] - 0s 550us/step - loss: 1.1005 - acc: 0.5167 - val_loss: 1.0097 - val_acc: 0.3500\n", - "Epoch 15/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 1.1293 - acc: 0.4167 - val_loss: 0.9818 - val_acc: 0.4000\n", - "Epoch 16/30\n", - "60/60 [==============================] - 0s 516us/step - loss: 1.1421 - acc: 0.4167 - val_loss: 0.9571 - val_acc: 0.4000\n", - "Epoch 17/30\n", - "60/60 [==============================] - 0s 533us/step - loss: 1.0784 - acc: 0.5000 - val_loss: 0.9312 - val_acc: 0.5500\n", - "Epoch 18/30\n", - "60/60 [==============================] - 0s 516us/step - loss: 1.0378 - acc: 0.4500 - val_loss: 0.9119 - val_acc: 0.6000\n", - "Epoch 19/30\n", - "60/60 [==============================] - 0s 533us/step - loss: 0.9446 - acc: 0.5000 - val_loss: 0.8883 - val_acc: 0.7500\n", - "Epoch 20/30\n", - "60/60 [==============================] - 0s 533us/step - loss: 0.9756 - acc: 0.5167 - val_loss: 0.8679 - val_acc: 0.7500\n", - "Epoch 21/30\n", - "60/60 [==============================] - 0s 633us/step - loss: 0.9451 - acc: 0.5667 - val_loss: 0.8540 - val_acc: 0.7500\n", - "Epoch 22/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 0.7960 - acc: 0.6500 - val_loss: 0.8393 - val_acc: 0.7500\n", - "Epoch 23/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 0.8670 - acc: 0.5833 - val_loss: 0.8223 - val_acc: 0.7500\n", - "Epoch 24/30\n", - "60/60 [==============================] - 0s 600us/step - loss: 0.9493 - acc: 0.6167 - val_loss: 0.8102 - val_acc: 0.7500\n", - "Epoch 25/30\n", - "60/60 [==============================] - 0s 566us/step - loss: 0.9749 - acc: 0.5500 - val_loss: 0.7922 - val_acc: 0.7500\n", - "Epoch 26/30\n", - "60/60 [==============================] - 0s 583us/step - loss: 0.9339 - acc: 0.6333 - val_loss: 0.7809 - val_acc: 0.8000\n", - "Epoch 27/30\n", - "60/60 [==============================] - 0s 550us/step - loss: 0.7419 - acc: 0.7000 - val_loss: 0.7721 - val_acc: 0.8000\n", - "Epoch 28/30\n", - "60/60 [==============================] - 0s 583us/step - loss: 0.8690 - acc: 0.6167 - val_loss: 0.7634 - val_acc: 0.8000\n", - "Epoch 29/30\n", - "60/60 [==============================] - 0s 583us/step - loss: 0.8348 - acc: 0.6167 - val_loss: 0.7556 - val_acc: 0.8000\n", - "Epoch 30/30\n", - "60/60 [==============================] - 0s 533us/step - loss: 0.7058 - acc: 0.7000 - val_loss: 0.7460 - val_acc: 0.8000\n" - ] - } - ], - "source": [ - "#建立優化防止梯度下降訓練\n", - "from keras import models\n", - "from keras import layers\n", - "from keras import optimizers\n", - "\n", - "model = models.Sequential()\n", - "model.add(layers.Dense(256, activation='relu', input_dim=3 * 3 * 512))\n", - "model.add(layers.Dropout(0.5))\n", - "model.add(layers.Dense(3, activation='softmax'))\n", - "\n", - "model.compile(optimizer=optimizers.RMSprop(lr=2e-5),\n", - " loss='categorical_crossentropy',\n", - " metrics=['acc'])\n", - "\n", - "history = model.fit(train_features, train_labels,\n", - " epochs=30,\n", - " batch_size=100,\n", - " validation_data=(validation_features, validation_labels))" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": {}, - "outputs": [], - "source": [ - "model.save('train3.h5')" - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtAAAAEjCAYAAAAbuTa2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3iUVfbA8e9JIwQSIA2EAEFqACkhUgTpsiIqIqyCYEGxrW2Lrui6rrr6s3dcFRVsCBYUcRVRuopSQhNIqAYJLSEBAoEASc7vj3eSDZAygZlMEs7neeYh88597z0zCTNn7nuLqCrGGGOMMcYY9/j5OgBjjDHGGGOqEkugjTHGGGOMKQdLoI0xxhhjjCkHS6CNMcYYY4wpB0ugjTHGGGOMKQdLoI0xxhhjjCkHS6CNMWctEZklItf7Og5fEMdkEdknIkt9HY8xxlQllkAbYyo9EUkRkSMicsiV8H0tIo3PtF5VHayq751GPCIid4vIWhHJFpFUEflURM5zPf6uiKiIdC1yTgsR0SL3F4hITtHnISIDRSSllHbV1d4hEdkhIi+IiH9543fpBVwExKhq17IKG2OM+R9LoI0xVcVlqlobOAfYA7zqw1heBu4B7gbCgVbADGBIkTKZwONl1JMN/LOcbXd0vQ4DgGuAm8t5PiISADQFUlQ1+zTPN8aYs5Yl0MaYKkVVc4DPgLYFx0RkiIisFJEsEdkuIo8UeSxYRD4UkQwR2S8iy0SkvuuxBSIyrkjZm0UkSUQOish6EYk/uX0RaQncAYxS1XmqelRVD6vqFFV9qkjR94AOItKnlKfzCjBKRFqcxuuQDPwAtHfF1VBEpotIuoj8JiJ3F4n5ERH5zPU6ZAE3AW8DPVy92Y8Wef6bRSRTRGaKSMMidaiI3CEim4BNRY79SUQ2uV6zf4tIcxH52fW7+EREglxl64nIf13x7XP9HFOk/gWu839y1fWdiEQWebyXiCx2/Q63i8gNruM1ROQ5EfldRPaIyBsiUrO8r6cxxpSHJdDGmCpFREKAq4FfihzOBq4D6uL0At8uIle4HrseqAM0BiKA24AjxdT7R+ARVz1hwOVARjEhDABSVbWsccOHgf8DniilzA7gLVe75SIibYELgZUi4gd8BawGGrli/LOI/KHIKUNxvnjUBd7HeR1+VtXaqvovEekPPAlchdPLvw2YdlKzVwDdKPLlBbgY6AJ0B/4OTARG47ze7YFRrnJ+wGScnu8mOL+DCSfVfw0wFogGgoB7Xc+1CTAL56pDFNAJWOU652mcKwCdgBau5/9waa+dMcacKUugjTFVxQwR2Q9k4YzdfbbgAVVdoKq/qmq+qq4BpgIFPb/HcRLnFqqap6qJqppVTP3jgGdUdZk6NqvqtmLKRQC73Iz5TaCJiAwupcyTwGUi0s7NOleIyD6chPltnKT0fCBKVR9T1WOquhUnMR9Z5LyfVXWG6zU65QsETtI7SVVXqOpR4AGcHurYorGqauZJ5z+tqlmqug5YC3ynqltV9QBO0tsZQFUzVHW6q7f+IM4Xi5N75yer6kZX/Z/gJMUFsc1R1amqetxV1yoREZwhLH9xxXUQ50vLSIwxxotsHJsxpqq4QlXnuCbNDQUWikhbVd0tIt2Ap3B6PIOAGsCnrvM+wOkNnSYidYEPgX+o6vGT6m8MbHEjjgycHtoyqepREfk38G/+1xN7cpl0EZkAPAa87ka18aq6uegBEWkKNHR9wSjgjzPEo8D2MuptCKwoEtchEcnA6dFNKaWOPUV+PlLM/QauGEOAF3F6rOu5Hg8VEX9VzXPd313k3MNAbdfPJf1uooAQINHJpQEQnOdujDFeYz3QxpgqxdWL/DmQh7OSBMBHwEygsarWAd7ASaRw9Vg+qqptgQuAS3GGaZxsO9DcjRDmAjEikuBmyJNxhpAMK6XMs0A/nKEQp2M78Juq1i1yC1XVS4qU0ZJOdtmJM7wCABGphdPbvqMcdZTmb0BroJuqhgG9C5py49ySfjd7cZL0dkWedx3XJEtjjPEaS6CNMVWKOIbi9GImuQ6HApmqmuNaOu6aIuX7ich5rp7rLJwhHXkn14szHOJeEeniaqOFq2f3BKq6CfgPMFVE+opIkGui4kgRGV9M+VycMc73l/ScVHU/8DzOGOLTsRTIEpH7RaSmiPiLSHsROb8cdXwEjBWRTiJSA2coxBJVTTnNmE4WipPs7heRcOBf5Th3CjBQRK4SkQARiRCRTqqajzNU5UURiQYQkUYnjf02xhiPswTaGFNVfCUih3CS4CeA613jbgH+BDwmIgdxJpB9UuS8BjiT57JwEu6FOMM4TqCqn7rq/Qg4iLMsXXgJsdyNMwHuNWA/zvCCYTjjkoszlbLHTb9M8Yl9mVxDIC7DGTP8G07P7Ns4Pd/u1jEXZ0m96a5Ym+PZscQvATVdsf0CfFuO2H4HLsHpxc7EmUDY0fXw/cBm4BfXCiNzcHq6jTHGa0T1TK7IGWOMMcYYc3axHmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhT6YjIAhHZJyI1fB2LMcaYykFEUkRkoK/jMAYsgTaVjIjEAhcCClxege0GVFRbxhhjjKnaLIE2lc11wC/Au8D1BQdFpKaIPC8i20TkgIj8KCI1XY/1EpHFIrJfRLaLyA2u4wtEZFyROm4QkR+L3FcRuUNENgGbXMdedtWRJSKJInJhkfL+IvKgiGwRkYOuxxuLyGsi8nzRJyEiX4nIn73xAhljjPkfEblZRDaLSKaIzBSRhq7jIiIvikia63NjjYi0dz12iYisd72X7xCRe337LExVYwm0qWyuA6a4bn8Qkfqu488BXYALgHDg70C+iDQBZgGvAlFAJ2BVOdq7AugGtHXdX+aqIxz4CPhURIJdj/0VGAVcAoQBNwKHgfeAUSLiByAikcAAYGp5nrgxxpjyEZH+wJPAVcA5wDZgmuvhQUBvoBVQF7gayHA99g5wq6qGAu2BeRUYtqkGLIE2lYaI9AKaAp+oaiKwBbjGlZjeCNyjqjtUNU9VF6vqUWA0MEdVp6rqcVXNUNXyJNBPqmqmqh4BUNUPXXXkqurzQA2gtavsOOAhVd2gjtWuskuBAzhJM8BIYIGq7jnDl8QYY0zpRgOTVHWF6zPhAaCHazjgcSAUaAOIqiap6i7XeceBtiISpqr7VHWFD2I3VZgl0KYyuR74TlX3uu5/5DoWCQTjJNQna1zCcXdtL3pHRP4mIkmuy337gTqu9stq6z1gjOvnMcAHZxCTMcYY9zTE6XUGQFUP4fQyN1LVecAE4DVgj4hMFJEwV9HhOFcTt4nIQhHpUcFxmyrOEmhTKbjGM18F9BGR3SKyG/gL0BHnslwO0LyYU7eXcBwgGwgpcr9BMWW0SAwXAve74qinqnVxepbFjbY+BIaKSEcgDphRQjljjDGesxPnyiUAIlILiAB2AKjqK6raBWiHM5TjPtfxZao6FIjGeb/+pILjNlWcJdCmsrgCyMMZi9zJdYsDfsAZFz0JeEFEGrom8/VwLXM3BRgoIleJSICIRIhIJ1edq4ArRSRERFoAN5URQyiQC6QDASLyMM5Y5wJvA/8WkZauySkdRCQCQFVTccZPfwBMLxgSYowxxqMCRSS44IaT+I4VkU6uz4T/A5aoaoqInC8i3UQkEKdDJQfIE5EgERktInVU9TiQhfP5Y4zbLIE2lcX1wGRV/V1VdxfccC6/jQbGA7/iJKmZwNOAn6r+jnMZ7m+u46tweq0BXgSOAXtwhlhMKSOG2TgTEjfiXBLM4cQhHi/gvFl/h/OG+w5Qs8jj7wHnYcM3jDHGW74BjhS5XQj8E5gO7MK5SjjSVTYMeAvYh/OenoEzIR3gWiBFRLKA2/jfEDxj3CKqWnYpY0yZRKQ3zlCOWFXN93U8xhhjjPEO64E2xgNclwjvAd625NkYY4yp3iyBNuYMiUgcsB9nsuNLPg7HGGOMMV5mQziMMcYYY4wpB+uBNsYYY4wxphwCfB1AeUVGRmpsbKyvwzDGnOUSExP3qmqUr+Ooruy93hhTGZT0Xl/lEujY2FiWL1/u6zCMMWc5EdlWdilzuuy93hhTGZT0Xm9DOIwxxhhjjCkHS6CNMcYYY4wpB0ugjTHGGGOMKYcqNwa6OMePHyc1NZWcnBxfh2IqkeDgYGJiYggMDPR1KMYYY84Clo9UXeXNGbyaQIvIxcDLgD/ODm1PnfR4E+A9oK6rzHhV/aa87aSmphIaGkpsbCwi4oHITVWnqmRkZJCamkqzZs18HY4xxpizgOUjVdPp5AxeG8IhIv7Aa8BgoC0wSkTanlTsIeATVe0MjAT+czpt5eTkEBERYX+sppCIEBERYb0AxhhjKozlI1XT6eQM3uyB7gpsVtWtACIyDRgKrC9SRoEw1891gJ2n25j9sZqT2d9EJaQK25dCbiX7YlOzHpzTwddRmNOUn6+s2XGAOjUDaRZZy9fhmLOcffZUTeX9vXkzgW4EbC9yPxXodlKZR4DvROQuoBYwsLiKROQW4BaAJk2aeDxQY0wFWT0VZtzu6yhO1bw/XPuFr6MwpyknN4+r3vyZ0d2a8K/L2vk6HGPMWcCbCXRxqbyedH8U8K6qPi8iPYAPRKS9quafcJLqRGAiQEJCwsl1+FxGRgYDBgwAYPfu3fj7+xMV5Wxas3TpUoKCgsqsY+zYsYwfP57WrVuXq+0hQ4aQlZXFDz/8UP7AjalIebmw8BlocB5c/LSvozlRzbq+jsCcgZCgAHo2j2BuUhoPX9rWegDNWcsX+cjbb7/N2rVreemll04/8CrImwl0KtC4yP0YTh2icRNwMYCq/iwiwUAkkObFuDwuIiKCVatWAfDII49Qu3Zt7r333hPKqCqqip9f8cPOJ0+eXO52MzIy+PXXXwkODub333/3Wu98bm4uAQHVYsEW40u/fgL7foORH0FsT19HYzxARBoD7wMNgHxgoqq+fFIZwZlMfglwGLhBVVd4Opb+cfWZv2EtW9IP0SI61NPVG1Ml+CofORt5cx3oZUBLEWkmIkE4kwRnnlTmd2AAgIjEAcFAuhdjqlCbN2+mffv23HbbbcTHx7Nr1y5uueUWEhISaNeuHY899lhh2V69erFq1Spyc3OpW7cu48ePp2PHjvTo0YO0tOK/T3z22WdcccUVXH311Xz88ceFx3fv3s3QoUPp0KEDHTt2ZMmSJYDzn6Lg2NixYwEYM2YMM2bMKDy3du3aAMyZM4eBAwcycuRIOnfuDMBll11Gly5daNeuHW+//XbhOV9//TXx8fF07NiRQYMGkZeXR4sWLcjMzAQgLy+Pc889t/C+OQvl5cKiZ53e59aX+Doa4zm5wN9UNQ7oDtxRzGTxwUBL1+0W4HVvBDKgTTQAc5OqVP+LMRXC2/lIcT788EPOO+882rdvz4MPPgg4HXLXXntt4fFXXnkFgBdffJG2bdvSsWNHxowZ49kn7yVe61ZU1VwRuROYjbNE3SRVXScijwHLVXUm8DfgLRH5C87wjhtU9YyGaDz61TrW78w60/BP0LZh2GmPq1u/fj2TJ0/mjTfeAOCpp54iPDyc3Nxc+vXrx4gRI2jb9sTPmwMHDtCnTx+eeuop/vrXvzJp0iTGjx9/St1Tp07lySefpE6dOowZM4b77rsPgDvuuIOLLrqIO++8k9zcXA4fPszq1at5+umnWbx4MeHh4W4ls7/88gvr168v7Nl+7733CA8P5/DhwyQkJDB8+HCOHj3K7bffzg8//EDTpk3JzMzE39+fUaNG8dFHH3HnnXcye/Zszj//fMLDw0/rNTTVwK+fQuZWuHoK2OX1akNVdwG7XD8fFJEknPkvRSeLDwXed723/yIidUXkHNe5HtOwbk3izgljblIat/Zp7smqjTktZ1M+crLU1FQeeughli9fTp06dRg4cCD//e9/iYqKYu/evfz6668A7N+/H4BnnnmGbdu2ERQUVHissvPqToSq+o2qtlLV5qr6hOvYw67kGVVdr6o9VbWjqnZS1e+8GY8vNG/enPPPP7/w/tSpU4mPjyc+Pp6kpCTWr19/yjk1a9Zk8ODBAHTp0oWUlJRTyuzYsYPff/+d7t2707ZtW/Ly8khOTgZgwYIF3HrrrQAEBAQQFhbGvHnzuPrqqwuTWHeS2R49epwwLOTFF18s/BaamprKli1b+Pnnn+nXrx9NmzY9od6bbrqJ9957D4BJkyYV9nibs1BB73P986DNEF9HY7xERGKBzsCSkx4qbkJ5oxLquEVElovI8vT08l+MHBgXzfJtmew/fKzc5xpT3XkrHynOkiVL6N+/P5GRkQQGBnLNNdewaNEiWrRowYYNG7jnnnuYPXs2derUAaBdu3aMGTOGKVOmVJnNz6rdwNbKNgO7Vq3/Lam0adMmXn75ZZYuXUrdunUZM2ZMsWsOFh3k7+/vT25u7illPv74YzIyMgoX/D5w4ADTpk3jkUceAU5djkVVi51YExAQQH6+M2czLy/vhLaKxj5nzhwWLVrEL7/8Qs2aNenVqxc5OTkl1hsbG0u9evWYP38+K1euZNCgQcW+PuYssHY6ZG6Bqz6w3udqSkRqA9OBP6vqyV1u7kwodw6e4YTx/m2ieXXeZhZsSOeKzsXm6MZUmLMlHylOSYMJIiIiWLNmDbNmzeKVV15h+vTpTJw4kdmzZ7Nw4UK+/PJLHn/8cdauXYu/v385n2HF8moPtDlRVlYWoaGhhIWFsWvXLmbPnn3adU2dOpU5c+aQkpJCSkoKS5cuZerUqQD069ev8BJNXl4eWVlZDBw4kGnTphUO3Sj4NzY2lsTERAC++OIL8vLyim3vwIEDhIeHU7NmTdatW8eyZcsA6NmzJ/PmzWPbtm0n1AtOL/To0aMZOXJkiZMVTDWXn+f0Pke3gzaX+joa4wUiEoiTPE9R1c+LKeLOhHKP6BhTl8jaQcxNtnHQxpTGk/lIcbp37878+fPJyMggNzeXadOm0adPH9LT01FV/vjHP/Loo4+yYsUK8vLySE1NpX///jz77LOkp6dz+PBhj8bjDdWuB7oyi4+Pp23btrRv355zzz2Xnj1PbyWCLVu2sHv3bhISEgqPtWzZkho1apCYmMiECRO4+eabefPNNwkICODNN9+ka9eu/P3vf6d3794EBATQpUsX3nnnHW699VaGDh3K999/z6BBg6hRo0axbQ4ZMoSJEyfSsWNH2rRpQ7duzpLe9evX5/XXX2fo0KGoKg0bNmTWrFkADBs2jBtvvJEbbrjhtJ6nqQbWfg4Zm+CP74F9iap2XCtsvAMkqeoLJRSbCdzp2kyrG3DA0+OfC/j5Cf1aR/Ptut0cz8sn0N/+5owpjqfykQLvvPMOn332WeH95cuX89hjj9G3b19Ulcsuu4whQ4awYsUKbrrppsKr108//TS5ublcc801HDx4kPz8fO6//35CQyv/SjpyhnP2KlxCQoIuX778hGNJSUnExcX5KCJTkl9++YUHHniA+fPn+ywG+9vwofw8eK0b+AfCbT9VuwRaRBJVNaHsktWXiPQCfgB+xVnGDuBBoAmAqr7hSrIn4CxZehgYq6rLi6nuBMW917vj27W7ue3DRKbe3J0ezSPKfb4xZ8I+c6q24n5/Jb3XWw+08YonnniCiRMnMm3aNF+HYnxl3Reu3ud3q13ybByq+iPFj3EuWkaBOyomIriwZSRB/n7MTdpjCbQxxmvsU814xT/+8Q+2bdtGjx49fB2K8YX8PGfXwag4iBvq62jMWaRWjQC6N49gno2DNsZ4kSXQxhjPWz8D9m6APvdZ77OpcAPaRLN1bzZb0w/5OhRjTDVln2zGGM/Kz3f1PreBtlf4OhpzFurv2pXQeqGNMd5iCbQxxrPWz4D0ZOh9H/hV7nU8TfXUODyE1vVDmZO0x9ehGGOqKUugjTGeU9D7HNkK2g3zdTTmLDYgLpplKfs4cOS4r0MxxlRDlkB7QN++fU9ZhPyll17iT3/6U6nn1a5dG4CdO3cyYsSIEusuaymnl1566YRFxy+55BKP7iXfsWNHRo0a5bH6TDWWNBPSk6D336332fjUgLho8vKVhRvLvyW4MVVVdc1HHnnkEZ577rkzrseTLIH2gFGjRp2yXNu0adPcTjobNmx4wgLk5XXyH+w333xD3bp1T7u+opKSksjPz2fRokVkZ2d7pM7iuLs9qKnECnqfI1pC+yt9HY05y3VqXI/wWkHMs2Ec5ixSnfORysbWgfaAESNG8NBDD3H06FFq1KhBSkoKO3fupFevXhw6dIihQ4eyb98+jh8/zuOPP87QoScu65WSksKll17K2rVrOXLkCGPHjmX9+vXExcVx5MiRwnK33347y5Yt48iRI4wYMYJHH32UV155hZ07d9KvXz8iIyOZP38+sbGxLF++nMjISF544QUmTZoEwLhx4/jzn/9MSkoKgwcPplevXixevJhGjRrx5ZdfUrNmzVOe20cffcS1115LUlISM2fOLPxPuHnzZm677TbS09Px9/fn008/pXnz5jzzzDN88MEH+Pn5MXjwYJ566in69u3Lc889R0JCAnv37iUhIYGUlBTeffddvv76a3JycsjOzmbmzJklvlbvv/8+zz33HCJChw4d+M9//kOHDh3YuHEjgYGBZGVl0aFDBzZt2kRgYKC3ftVn7nAmvDsEjnjuCkGloXlwaA9c+Zb1Phuf8/cT+raOYm5SGrl5+QTYroTmLFCd85HiFFdndnY2V111FampqeTl5fHPf/6Tq6++mvHjxzNz5kwCAgIYNGjQGfdoV78EetZ42P2rZ+tscB4MfqrEhyMiIujatSvffvstQ4cOZdq0aVx99dWICMHBwXzxxReEhYWxd+9eunfvzuWXX46zOdepXn/9dUJCQlizZg1r1qwhPj6+8LEnnniC8PBw8vLyGDBgAGvWrOHuu+/mhRdeYP78+URGRp5QV2JiIpMnT2bJkiWoKt26daNPnz7Uq1ePTZs2MXXqVN566y2uuuoqpk+fzpgxY06J5+OPP+b7779nw4YNTJgwoTCBHj16NOPHj2fYsGHk5OSQn5/PrFmzmDFjBkuWLCEkJITMzMwyX9qff/6ZNWvWEB4eTm5ubrGv1fr163niiSf46aefiIyMJDMzk9DQUPr27cvXX3/NFVdcwbRp0xg+fHjlTp4BdiRC2npocynUrOfraDyvdjS0H+7rKIwBYGBcfT5fsYMVv++na7NwX4djzjaWjxTyRD5yspLq3Lp1Kw0bNuTrr78G4MCBA2RmZvLFF1+QnJyMiHhkWEn1S6B9pOCyScEfbME3IlXlwQcfZNGiRfj5+bFjxw727NlDgwYNiq1n0aJF3H333QB06NCBDh06FD72ySefMHHiRHJzc9m1axfr168/4fGT/fjjjwwbNoxatWoBcOWVV/LDDz9w+eWX06xZMzp16gRAly5dSElJOeX8ZcuWERUVRdOmTYmJieHGG29k3759BAQEsGPHDoYNcyaJBQcHAzBnzhzGjh1LSEgIAOHhZX9gXXTRRYXlSnqt5s2bx4gRIwr/QxaUHzduHM888wxXXHEFkydP5q233iqzPZ9LT3b+vfxVCLEPdGM84lg2/PgSNL0AmvcrPHxhy0gC/YW5SXssgTZnjeqYj5Snzosvvph7772X+++/n0svvZQLL7yQ3NxcgoODGTduHEOGDOHSSy91q43SVL8EupRvZt50xRVX8Ne//pUVK1Zw5MiRwm9qU6ZMIT09ncTERAIDA4mNjSUnJ6fUuor7Nvjbb7/x3HPPsWzZMurVq8cNN9xQZj3ODrrFq1GjRuHP/v7+J1yaKTB16lSSk5OJjY0FICsri+nTp3PVVVeV2F5xsQcEBJCfnw9wSswFf/hQ8mtVUr09e/YkJSWFhQsXkpeXR/v27Ut8vpVGWjLUirbk2RhP8g+ClR9C6rITEujQ4EC6NYtgbnIaD1wS58MAzVnJ8pFCZ5qPlKfOVq1akZiYyDfffMMDDzzAoEGDePjhh1m6dClz585l2rRpTJgwgXnz5rnVTklsUJiH1K5dm759+3LjjTeeMFj/wIEDREdHExgYyPz589m2bVup9fTu3ZspU6YAsHbtWtasWQM4yWutWrWoU6cOe/bsYdasWYXnhIaGcvDgwWLrmjFjBocPHyY7O5svvviCCy+80K3nk5+fz6effsqaNWtISUkhJSWFL7/8kqlTpxIWFkZMTAwzZswA4OjRoxw+fJhBgwYxadKkwgkEBUM4YmNjSUxMBCh1ckJJr9WAAQP45JNPyMjIOKFegOuuu45Ro0YxduxYt56Xz6UnQ3QbX0dhTPXiHwjn3whb50P6hhMe6t8mms1ph9iW4b1J0MZUJtUtHyktvuLq3LlzJyEhIYwZM4Z7772XFStWcOjQIQ4cOMAll1zCSy+9xKpVq86obbAE2qNGjRrF6tWrGTlyZOGx0aNHs3z5chISEpgyZQpt2pSePN1+++0cOnSIDh068Mwzz9C1a1fAWUquc+fOtGvXjhtvvJGePXsWnnPLLbcwePBg+vXrd0Jd8fHx3HDDDXTt2pVu3boxbtw4Onfu7NZzWbRoEY0aNaJRo0aFx3r37s369evZtWsXH3zwAa+88godOnTgggsuYPfu3Vx88cVcfvnlJCQk0KlTp8IB+vfeey+vv/46F1xwAXv37i2xzZJeq3bt2vGPf/yDPn360LFjR/7617+ecM6+ffuqxjJ7qs6He5Ql0MZ4XJex4F8Dlk484fCAOGdXwrlJtiuhOXtUp3ykwOOPP05MTEzhraQ6f/31V7p27UqnTp144okneOihhzh48CCXXnopHTp0oE+fPrz44ovlars4Ulq3emWUkJCgJ69DmJSURFycXZ47G3322Wd8+eWXfPDBB8U+Xqn+Ng6kwovtYMgLcP5Nvo7GnCERSVTVBF/HUV0V915fphl/gnUz4G9JEFyn8PDAFxZSP6wGU8Z193CUxpyoUn3mmHIr7vdX0nu99UCbKuuuuzKqnU0AACAASURBVO5i/Pjx/POf//R1KO4pmEBoPdDGeEfXW+B4NqyccsLhAXHRLNmaycEc25XQGOMZlkCbKuvVV19l8+bNtGrVytehuCfNEmhjvKphJ2jcHZa+Cfl5hYcHtKlPbr6yaGPJQ8iMMaY8qk0CXdWGohjvq3R/E+nJUCsKakX4OhJjqq9ut8K+FNj0feGh+CZ1qRsSyFzbldBUgEr32WPcUt7fW7VIoIODg8nIyLA/WlNIVcnIyChco7pSSE+23mdjvC3uMghtCEveKDwU4O9H31ZRzN+QRl6+fU4Y77F8pGo6nZyhWqwDHRMTQ2pqKunp6b4OxVQiwcHBxMTE+DoMR8EKHB2u9nUkxlRv/oHOJN15/3atetMagAFx9Zmxaicrf99HQqytw268w/KRqqu8OUO1SKADAwNp1qyZr8MwpmQHd8HRrMIPc2OqCxGZBFwKpKnqKbsZiUgd4EOgCc5nznOqOtmrQXW5ARY+4yxpN+R5AHq3iiLAT/g+aY8l0MZrLB85e1SLIRzGVHppSc6/NoTDVD/vAheX8vgdwHpV7Qj0BZ4XkSCvRlQrEs4bAaumwpH9ANSpGUjf1lF8smw7h4/lerV5Y0z1Zwm0MRWhYHe0aFsf1FQvqroIyCytCBAqzp7AtV1lvZ/BFixpt+p/S9rd3rc5+w4f56Mlv3u9eWNM9WYJtDEVIT0JQiKcnjFjzi4TgDhgJ/ArcI+q5nu91YadoEkPZxiHa0m7Lk3D6X5uOG/9sJWjuXllVGCMMSXzagItIheLyAYR2Swi44t5/EURWeW6bRSR/d6MxxifSd8AUdb7bM5KfwBWAQ2BTsAEEQkrrqCI3CIiy0VkuUcmYXW95ZQl7e7s15I9WUf5LDH1zOs3xpy1vJZAi4g/8BowGGgLjBKRtkXLqOpfVLWTqnYCXgU+91Y8xviMqmsJO5tAaM5KY4HP1bEZ+A0odjKAqk5U1QRVTYiKijrzlotZ0q5niwg6Nq7LGwu3kJvn/Y5wY0z15M0e6K7AZlXdqqrHgGnA0FLKjwKmejEeY3zj4G7IOWATCN2Ul698+Mu2Ctt2+cNftvH5CuuN9KLfgQEAIlIfaA1srZCWC5a02zq/cB6CiHBnvxZszzzCzNU7KyQMY0z1480EuhGwvcj9VNexU4hIU6AZMK+Exz17Wc+YipTu2sI72hJod8xN2sNDM9ZWyEQvVWXCvM18t852qDtdIjIV+BloLSKpInKTiNwmIre5ivwbuEBEfgXmAverasXtqd3lBvCv4YyFdhnQJpo2DUL5z4It5NvGKsaY0+DNBFqKOVbSO9VI4DNVLXZWh8cv6xlTkQoSaOuBdkvB2NS5yWleb2vdzix2Z+UwIC7a621VV6o6SlXPUdVAVY1R1XdU9Q1VfcP1+E5VHaSq56lqe1X9sEIDLGZJOz8/4U/9WrA57RDfrd9doeEYY6oHbybQqUDjIvdjcGZhF2ckNnzDVFfpyVAzHGrZl7+yZBw6yrzkNGrXCCBx2z72Hz7m1fbmJachAn1bWwJdrRWzpN2Q886hWWQtJszfbNsuG2PKzZsJ9DKgpYg0cy2aPxKYeXIhEWkN1MO5BGhM9ZO+wel9luIuypiiZq7eSW6+8tCQOPLylYUbvTtka25yGh1j6hIVWsOr7RgfK2ZJO38/4fY+zVm7I8vrf2fGmOrHawm0quYCdwKzgSTgE1VdJyKPicjlRYqOAqapdQGY6kjV2YXQVuBwy/QVqbRrGMZVCY2JqBXE3CTvDeNIO5jD6u37GWjDN84O3W49ZUm7Kzo3omGdYF6bv9l3cRljqiSvrgOtqt+oaitVba6qT7iOPayqM4uUeURVT1kj2phq4VAa5Oy3HQjdkLw7i7U7shjRJQY/P6Ffm2gWbEjz2lJjC5KdXsf+bep7pX5TybS5FMIawY8vOF9sgaAAP27pfS7LUvaxZGuGjwM0xlQlthOhMd6UnuT8az3QZZqemEqAn3B5x4YADIyLJisnl8Rt+7zS3tzkPTSsE0zcOaFeqd9UMv6B0OfvsH0JJP+38PDIrk2IrB3EBOuFNsaUgyXQxniTa+1Z24WwdLl5+Xyxcif920QTUdsZj9yrZRSB/uKV1Thyjufxw6a99I+LRmxs+tmj0xiIbA1zHoE8Z53x4EB/bup1Lj9s2svq7bYZrjHGPZZAG+NN6ckQXBdq2zjb0izalM7eQ0cZ3iWm8FjtGgF0PzeCuUmeX6N5yW+ZHD6WxwAbvnF28Q+Aix6FjM2w4r3Cw2O6NyEsOMDGQhtj3GYJtDHelJZsK3C4YXriDuqFBNLvpOXkBrSJZkt6Nil7sz3a3rykPQQH+tGjeYRH6zVVQKuLoWkvWPAUHD0IQGhwIDf0bMZ36/ewYfdBHwdojKkKLIE2xltUnTHQtgNhqfYfPsb36/cwtFMjggJOfEsqmODnyWEcqsqcpDR6tYgkONDfY/WaKkIELnoMstNh8auFh8deEEtIkD+vL7BeaGNM2SyBNsZbstPhyD7bgbAMX63ZxbG8fEYUGb5RoElECC2jazMv2XPDODbuOcSO/UcYEGfDN85aMV2g3ZVOAp21C4B6tYIY070pM1fvZFuGZ694GGOqH0ugjfEW28LbLdMTU2nTIJR2DcOKfbx/XDRLtmZyMOe4R9qb60rGTx4uYs4yAx52JhIueLLw0LhezQjw9+Ppb5O9tnyiMaZ6sATaGG8pXIHDEuiSbE47xKrt+xkeH1PiahgD4+qTm6/8sGmvR9qcm5RG+0ZhNKgT7JH6TBUV3gy63gwrP3DmKgDRYcHc1a8F3/y6m9FvLyH94FEfB2mMqawsgTbGW9KSoEYdCG3g60gqrekrUvH3E4Z2blhimc6N61I3JJA5HliNIzP7GCt+32erbxhH7/sgKNRZ1s7lrgEtefHqjqxO3c+lr/5A4rZM38VnjKm0LIE2xlvSNzgTCG0FjmLl5Sufr0ilT6sookNL7g0O8Pejb6soFmxIJy9fz6jNBRvSUIUBtn23AQgJhwv/AhtnQcqPhYeHdY7hiz/1JDjQn6vf/IX3FqegemZ/e8aY6sUSaGO8JT3JdiAsxU+b97In6yjD40+dPHiyAXH1ycw+xqoz3OhiblIaUaE1aN+wzhnVY6qRbrc5W3x/9xDk/2/cc9w5Ycy8sxd9W0fxr5nr+MvHqzh8LNeHgRpjKhNLoI3xhuy9cDjD5zsQ5hzP42hunk9jKMn0FamEBQe41Rvcu1UU/n5yRpuqHMvNZ9HGdAa0icbPz64KGJfAmtD/Idi5EtZ/ccJDdWoGMvHaBO4d1IovV+/kyv8s9via5MaYqskSaGO8IS3J+dfHPdDXTVrKHVNW+jSG4mTlHOfbtbu5vFNDt9ZirlMzkPNj6zHvDNaDXp6SycGjufRvY8M3zEk6XA31z4M5j0LuiRMH/fyEO/u35L2xXdmdlcNlE37k+/We3x3TGFO1WAJtjDdUgiXs9mTlsPS3TOYm72HXgSM+i6M436zZxdHcfLeGbxQY0KY+ybsPkrrv8Gm1OScpjaAAP3q1jDyt80015ufvbPG9fxssn1Rskd6tovjvXb1oFlmLm99fzitzN1VwkMaYysQSaGO8IX0D1AiDsJJXl/C2+a7eWlX4YuUOn8VRnM8SU2keVYtOjeu6fU7BUI/5p9ELrarMTd7DBc0jCAkKKPf55izQYgCc2w8WPg1Hih9rH1MvhE9u7cGwzo144fuNLNjguR0yjTFViyXQxnhDerIzfMOHK3DMTU6jUd2aJDStx2eJqZVmFYGUvdks37aP4V1KXvu5OOdG1aZZZC3mJJU/adm6N5ttGYcZYMM3TGkuesxJnn98ocQiwYH+PHnlebSIrs0Dn/9Kloc2+DHGVC1lJtAiUl9E3hGRWa77bUXkJu+HZkwVlp7s0+EbOcfz+HHTXgbERTOiSwxb07PPeAULT/l8RSoiMKxzo3Kf279NND9vySD7aPlWQyiYfNjPEmiPE5FJIpImImtLKdNXRFaJyDoRWViR8ZXLOR2g0zXw82uwZ12JxYID/Xl2RAf2ZOXw5DdJFRigMaaycKcH+l1gNlBwLXoj8GdvBWRMlZedAdnpPk2gf96awZHjefRvE80lHc4hONCP6StSfRZPgfx8ZfqKHfRqEck5dWqW+/wBcdEcy8vnp83l25VwblIabRqEElMvpNxtmjK9C1xc0oMiUhf4D3C5qrYD/lhBcZ2eQY9DcF2YeRfkl7yCTecm9bj5wnOZunQ7P3pol0xjTNXhTgIdqaqfAPkAqpoLVM51sYypDCrBBMJ5SWmEBPnT/dwIwoID+UO7BsxctZOc4779r7s0JZMd+48woov7kweLOj82nNAaAcwtxzCOA4ePs3zbPts8xUtUdRFQ2nZ91wCfq+rvrvKVe+BwSDgMfhp2JMKSN0ot+peLWnFuVC3un76GQ+W8KmKMqdrcSaCzRSQCUAAR6Q4c8GpUxlRlBQl0tG8SaFVlXnIavVpEFi4RNzw+hqyc3HIlnt4wZ/0egvz9GBh3eltpB/r70bt1FPM2pJHv5q6ECzamkZev9Lftu32lFVBPRBaISKKIXFdSQRG5RUSWi8jy9PT0CgzxJO2HQ6uLYd7jsC+lxGIFQzl2HjjCU7NsKIcxZxN3Eui/AjOB5iLyE/A+cJdXozKmKktPhqBQZ3czH0jefZAd+4+c0OPas0UkDcKCfT6MY15yGt2bR1CrxumvhDEwLpr0g0dZu9O97/HzktMIrxVUrhU/jEcFAF2AIcAfgH+KSKviCqrqRFVNUNWEqKioiozxRCIw5AUQf/jqHmcpmxJ0aRrOTT2b8eEvv7O4nEOLjDFVV5kJtKquAPoAFwC3Au1UdY23AzOmyvLxChwFm430a/2/BNrfTxgW34iFG9NJO5jjk7i2ph9i697sM14Jo0+raPwEt3rTc/PyWbAhnX6to/G33Qd9JRX4VlWzVXUvsAjo6OOYylanEVz0CGxdAKs+KrXo3wa1JjYihPs/X1OuCa4Hc46zcGN6pVkhxxjjPndW4bgOZwxbFyAeGFXaJThjznppvl2BY27SHjrG1CE6LPiE48PjY8jLV75cudMncRUk9me6E2B4rSDim9RjbnLZu8ElbtvHgSPHbfyzb30JXCgiASISAnQDqsZ4hy43QpMLYPaDcKjkL2w1g/x5ZkRHUvcd4dnZG8qsVlWZuXonA55fyPWTlvKfBVs8GbUxpgK4M4Tj/CK3C4FHgMu9GJMxVdfhTMhO89kW3nsPHWXl9v3FjvdtEV2bTo3r+mxN6LlJabSuH0rj8DNfCaN/XDRrd2Sx+0DpvenzktMI9BcutN0HvUZEpgI/A61FJFVEbhKR20TkNgBVTQK+BdYAS4G3VbXEJe8qFT8/uPwVOH4Evrmv1KJdm4VzfY9Y3l2cwpKtGSWW25p+iGvfWcrdU1dSPyyYAW2iee67Dcy3TVmMqVLKHIioqieMdxaROsAHXovImKos3dX7FB3nk+YXbEhHlRJ7XId3ieGfM9aybmcW7RvVcbveKUu2sWBDOm+O6YLfaQyFOHDkOMtSMrm597nlPrc4A+Pq88y3G/jDS4sICii5H+DA4eN0axZBaHCgR9o1p1LVUW6UeRZ4tgLC8bzIltDn7zDv35D0X4i7tMSif7+4NfOS0/j79DV8e09vagb5Fz6WczyP1+Zv5s2FW6kR6MdjQ9sxultTjuXmc+Xri7ln6kpm3tmL2MhaFfGsjDFn6HRm8hwGWno6EGOqhXTXlWkf9UDPS95D/bAatGsYVuzjl3doyL+/Ws9nialuJ9DLUzJ5+Mt15OUrv2zN4IIW5e/NXbQxndx89dhOgC2ja3PfH1qTuu9IqeVE4I+nuWSeMYV63gPrZsDXf4PYXlCz+AmpIUEBPD28A6Pe+oVnZ2/g4cvaAs728w/PXMv2zCMM69yIBy5pQ3SoM8SqZpA/E6/twmUTfuSWD5bz+Z96UvsMJtkaYypGmf9LReQrXEvY4Qz5aAt84s2gjKmy0jdAUG2o07jCmz6Wm8+ijXu5rGPDErfIrhMSyEVt6/Plqh08eElcqb23APuyj3HX1JU0qluTfYeP8Vli6mkl0POS06gXEkjnJvXKfW5xRIQ7+rXwSF3GlMk/0BnK8fYA+P5h5+cS9GgewbXdmzJ58W/EN63LV6t3MnvdHppH1eKjm7txQfNT//80Dg9hwqh4rpu0hPs+Xc1/RseXa5t7Y0zFc2cM9HPA867bk0BvVR3v1aiMqarSkiCylU9W4Fj6WyaHjuaW2cs7oksM+w4fL3PMpapy76eryTh0jNeuiefSDucwa+3ucm8YkZuXz/wNabYShqnaGsVDjztgxXvw2w+lFh0/uA2N6tbkzo9WsnBjOvf9oTWz7uldbPJcoFfLSB4YHMestbttUqExVYA7y9gtLHL7SVXdXkhWRC4WkQ0isllEik26ReQqEVkvIutEpPS1goyp7NI3+GwFjrnJe6gR4EfPMnqIL2wZSVRoDaYnlv5f+e0ffmNuchoPXtKG82LqMKJLDEeO5/HNr7vKFdfK7fvZf/g4/W0ljEpNRFqJyFwRWeu630FEHvJ1XJVK3wehXjP46m5nYmEJatUIYMI18Yzu1oTv/9KHO/q1KPNqD8C4C5txeceGNqnQmCqgxP/RInJQRLKKuR0UkayyKhYRf+A1YDDOsI9RItL2pDItgQeAnqraDvjzGT0bY3zpyD44tNsnOxCqKnOT0ujZIvKEiUvFCfD3Y1jnRsxLTiPj0NFiy6z8fR9Pf5vMxe0acP0FsQDEN6lHs8haZSbeJ5ublEaAn9C7lQ83xjDueAvn/fg4gGu9/5E+jaiyCQqBy16GzK0w//9KLdqpcV2eGHZeuVadERGeHt6BNg3CuGfqSlL2Zp9pxMYYLykxgVbVUFUNK+YWqqrFz1A6UVdgs6puVdVjwDRg6EllbgZeU9V9rjbtK7epuvasd/71QQ/0lvRD/J552O01lofHx5Cb76xFe7IDh49z50craVAnmKdHdCgciykiXNm5EUt+y2R75mG3Y5uXvIeuzcIJs5UwKrsQVV160rHyjdc5G5zbB+Kvh8WvQNJXHq++YFKhn59wywfLyz1kyhhTMdwZAw2AiESLSJOCmxunNAK2F7mf6jpWVCuglYj8JCK/iMjFJbR9i4gsF5Hl6enp7oZsTMVaPgkCa0HM+RXedMGufO4m0K0bhNK+URifndSbrKrc99lq9mTlMOGaeOrUPDHpvbJLDCK4vSX49szDbNxz6Iw3TzEVYq+INMc1aVxERgDlG69zthj8NDRKgM9vgZ2rPF59waTCzWmHuO/T1bZToTGVkDs7EV4uIpuA34CFQAowy426i5stdPK7QADOknh9gVHA2yJyyvpAqjpRVRNUNSEqyi4Dm0oofSOsnQ5db4aQ8Apvfm5yGnHnhNGwbk23zxkRH8O6nVkk7/7fiKx3F6fw3fo9jB/chk6NT12qq1HdmvQ4N4LPV+xw60N9bpKzW+DAuFM3djGVzh3Am0AbEdmBM6Tudt+GVEkF1oSRH0HNcJg6ErI8v7unTSo0pnJzpwf630B3YKOqNgMGAD+5cV4qUHQtrxjg5HeZVOBLVT2uqr8BG7A1pk1VtOgZ50P1grvKLuth+w8fI3HbPgaWc5Le5Z0aEegvhWOa16Tu5/++SWJgXDQ39WpW4nnD42P4PfMwy1L2ldnG3OQ0zo2qZZtDVAGu4XYDgSigjar2UtUUH4dVeYXWh2s+hqMHYeooOOb+sCZ3FZ1UOKuck3eNMd7lTgJ9XFUzAD8R8VPV+UAnN85bBrQUkWYiEoQzGWXmSWVmAP0ARCQSZ0jHVrejN6Yy2LvJ6X0+fxzUqvgtoxduTCcvX8s9TCK8VhD920Tzxcqd7Ms+xp0frSSqdg2e+2PHUtegHXxeA2oF+fNZ4vYSywAcOprLkq2ZHts8xXiXiDwsIg8DfwP+UuS+KUmD9jD8bdi1Gr64FfLzPVp9waTC+Cb1uHvaSuYn2zQhYyoLdxLo/SJSG1gETBGRl3FjYomq5gJ3ArOBJOATVV0nIo+JyOWuYrOBDBFZD8wH7nMl68ZUHYuehYBguOBunzQ/NymNyNpBdIwpfne00gyPj2HvoaOMeGMxO/Yf4dVrOlM3JKjUc0KCAhh83jl88+tuDh8r+a3gx03pHMvLZ4AN36gqsovc8nBWUIr1ZUBVQuvBMOhxSJoJ8x/3ePU1g/yZPPZ82jQI49YPE1m8ea/H2zDGlJ87CfRQ4AjwF+BbYAtwmTuVq+o3qtpKVZur6hOuYw+r6kzXz6qqf1XVtqp6nqpOO72nYYyP7N0Mv34KCTdC7Yofn5+bl88C1yYlfqexSUm/NtFE1ApiS3o29/2hNV2aujd+e0SXGA4dzWX2ut0llpmblEZYcABdmnpm90HjXar6fJHbEzhzU06e+G2K0+MOZ2WOH56HVVM9Xn1YcCDv39iVZhG1GPf+cpanZHq8DWNM+ZS2DvQEEblAVbNVNU9Vc1X1PVV9xXqJjXFZ9Cz414Ce9/ik+eXb9pGVk8uA09ykJNDfj3v/0JrrezTllgvPdfu8rrHhxNSryfTEHcU+np+vzN+QRp/W0QT6u73Yj6lcQgD3/yjOZiIw5HmIvdDZZGXbzx5vol6tID4Y15UGYcGMnbyMNan7Pd6GMcZ9pX2ybQKeF5EUEXlaRNwZ92zM2SNjC/z6CZx/E9T2zTjfeclpBPn70avl6fd+j+rahEeHti9XD7afn3BlfAw/bdnLzv2n7si2OnU/ew8dK/fERuM7IvKriKxx3dbhTOp+2ddxVRn+gXDV+1CnMXw8GjJ/83gT0aHBTLm5G3VCArlu0tITVtAxxlSs0jZSeVlVewB9gExgsogkuSaWtKqwCI2prBY9B/5BPhv7DM4ycd3ODad2jYAKb3t4fCNU4YuVp/ZCz0tOw0+gj+0+WJVcijM87zJgENBQVSf4NqQqJiQcrvkE8vPgo6sh54DHmzinTk0+Gted4AB/xry9lK3phzzehjGmbGVeW1XVbar6tKp2Bq4BhuFMCjTm7JW5FdZ87Ix9DvXNJLmUvdlsSc/22SoXTSNq0TU2nOmJqaesCT03KY2EpuFlTkg0vici4SISDhwscjsChLmOm/KIbAFXfwCZW+DTsZDn+Z0Em0SE8OG4bqgqo99eUq6dQY0xnuHORiqBInKZiEzB2UBlIzDc65EZU5ktet65ZOujsc/grLEM0L+N71a5GNElhq17s1m5/X/jMXfuP8L6XVmnPS7bVLhEYLnr35Nvy30YV9XVrDcMeQG2zIVv7gUv7CTYIro2H47rxuFjeVzz9i/sOnDqUCpjjPeUNonwIhGZhLPZyS3AN0BzVb1aVWdUVIDGVDqZW2H1VOgyFkIb+CyMecl7aBldmyYRIT6LYfB5DQgO9DthS/B5rsTeEuiqQVWbqeq5rn9PvtkkwtPV5Xro+WdInAyLX/FKE3HnhPH+jV3Zl32c0W8vYe+ho15pxxhzqtJ6oB8EfgbiVPUyVZ2iqtkVFJcxldcPz4NfgE97nzenHWLxlgwGt/ddAg8QGhzIxe0a8NXqneQczwOcBLpJeAjNo2r7NDZTfiJST0S6ikjvgpuvY6rSBvwL2g2D7x+Gdd7pd+rYuC6Tx55P6r4jPPrVeq+0YYw5VWmTCPup6luqagtOGlNgXwqsngZdboCwc3wWxqvzNhEc4M/1F8T6LIYCI7o05mBOLt+v38ORY3n8tHkvA+KiS93N0FQ+IjIOZ8Os2cCjrn8f8WVMVZ6fH1zxOsR0dXYq3L7MK82cHxvO7X2a89XqnSzeYhutGFMRbIFWY8rjh+dB/KDXn30Wwpb0Q3y1eifX9WhKRO0aPoujQI/mEZxTJ5jpK1L5afNejubmM8CH47LNabsHOB/Ypqr9gM5AelknicgkEUkTkbVllDtfRPJEZIRnwq0iAmvCqKnOcK+pI72yvB3A7X2b0zi8Jv/6ch3H8zy7pbgx5lSWQBvjrn3bYNVHrt7nhj4LY8K8zdQI8Ofm3pVjeKq/nzCscyMWbUxn2rLfqV0jgK7NbPGGKihHVXMARKSGqiYDrd04713g4tIKiIg/8DROr/bZp1YkjP4M8nPho6vgyD6PNxEc6M/Dl7ZjU9oh3luc4vH6jTEncmcVjjtFxPbiNaag97mn73qft6Yf4stVO7i2R1MiK0Hvc4HhXWLIV5iTlEbvVpEEBdh38yooVUTqAjOA70XkS2BnWSep6iKcvQJKcxcwHUg74yirqsiWMHKK0wP98bWQe8zjTQyMi6Zv6yhemrOJtKwcj9dvjPkfdz7lGgDLROQTEblYbGCjORvt/x1WTYH466BOI5+FMWH+ZoIC/Li5HNtuV4TmUbXp3KQu4Ntl9czpU9VhqrpfVR8B/gm8A1xxpvWKSCOc/QPecKPsLSKyXESWp6eXOXqk6ontBUNfg5QfYOZdHl/eTkR45LJ2HMvN58lZyR6t2xhzInc2UnkIaInzZnoDsElE/k9Emns5NmMqj3VfOJdffdj7/NvebGas3MGYbk2JCq08vc8Fru8RS2iNAPq1tt0HqxIR+VpERotIrYJjqrpQVWeqqie6SV8C7lfVvLIKqupEVU1Q1YSoqGr6d9Txauj7IKyZBguf9nj1sZG1uKX3/7N33+FRldkDx79vKqR3CEkgCQkQIBB6EZSmgKggKkhT7B3Xsq7uz3Vd3bXt2kERGyoiiiJYUJEOIp3QkgChpRCSUEJCenl/f9yJBkiZSWZSz+d55oFM7tx7CHBzcua85w3n212pbD0qMwCEsBWz3mfVxjZjJ02PEsAb+Fop9YoNYxOi8chIAPdA8AppsBBmr07E0d6Ou69oXNXnchN6BbHzmSsbxcJGYZF5GNt4H1NKfamUmqCUsuYWkn2BRUqpY8CNwDtKqTpXtpu0K56AnlNh7YvGVB8re2B4BEFerXlm2T5KivwVswAAIABJREFUZEGhEDZhTg/0LKXUDuAV4DcgWmt9H9AH2ZFQtBSZ8eBvznoq2zh+OpelsalMH9iBAPdWDRZHTRztpfe5qdFaL9NaTwHaA0uAW4Ek03SNK61w/jCtdajWOhT4Gri/xW/GpRRc+yaEDoVlD0LiSquevrWTPf+4JoqEkzl8tvm4Vc8thDCY893OD5iotR6ttV6stS4G0FqXYVQthGjeysog8yD4RzVYCLNXJ+Jgp7inkVafRdOntc7XWn+ptb4euApjjN3PNb1OKfUFxqZbnZVSKUqpO5RS9yql7rVxyE2bgxNMXgD+XWDRdDi+yaqnH92tLUMj/XhtxUEyc2SHQiGszZwEejkVVlgrpdyVUgMAtNbxtgpMiEYjOwWKcxusAp10Oo8lu1KZOqB9o64+i6ZNKdVGKfWQUuo3jEkcKzDeaayW1nqK1jpQa+2otQ7WWn+otZ6rtb5k0aDWeqbW+msbhN80tfaCGd+CZzB8PglSd1rt1Eopnr2uGwUlpbwkCwqFsDpzEuh3gfMVPs41PSdEy5Bh+ubj36VBLj97zSHs7RT3XiHrdoX1KaXuUkqtBnYCnYAntNbhWuu/aa1jGzi85s/NH25ZBi7esGAipFtvO+6O/m7cMSScb3amsOO4LCgUwprMSaCVaREh8EfrhoPtQhKikcksT6DrvwKdfCaPJTtTmdq/PW08pPosbGIw8BIQorV+SGv9W0MH1OJ4BhlJtL0zfDYBTh+22qkfGhFBoGcrnlm2n9Iy647NE6IlMyeBPmJaSOhoejwMHLF1YEI0GpkJ4NYGXOp/d705axKxs1PcN0yqz8I2tNa3aa1XmIojoqH4hBtJdFkJfDoespKtclpXZwf+b1wU+09ks3CLLCgUwlrMSaDvxahQpAIpwADgblsGJUSjkpnQIO0byWfy+HpHClP6hUj1WYiWIKALTF8CBdlGEp2TbpXTjosOZHBHX/77ywFOn5cFhUJYgzkbqWRorW/WWgdordtoradqrVvudqyiZdEaMg80SAL9ztrD2CnFvVJ9FqLlaBcD0xZDzkmjnSOv7r3LSin+dV038opKeWrJXsqklUOIOjNnDnQrpdQDSql3THNBP1JKfVQfwQnR4M6lQNH5eu9/Tjmbx+LtyUzuF0KgZ+t6vbZomZRSHZVSzqbfDzO17nk1dFwtUvsBMGWh0Qu9YKJRka6jyDbuPDm2Cyvi0nl7daIVghSiZTOnheMzoC0wGlgHBAM5tgxKCHPkFZVQUFzj7sB1k3nA+DWgfmdAl1efpfdZ1KNvgFKlVATwIRAGLGzYkFqw8GEw6RM4uRcWToai3Dqf8o4hYUzsHcTrKw/yy/6TdT6fEC2ZOQl0hNb6H0Cu1voTYBwQbduwhKheaZlm4jubeOyr3ba9UKZp1Hk9tXAUlZTx6ooDfLktmUn9gmnnJdVnUW/KtNYlwPXAG1rrR4DABo6pZes8FibOg+TNRk90Hds5lFK8cH00PYM9efTLWA6clFqYELVlTgJdbPo1SynVHfAEQm0WkRBm+GHPCRJO5rD+YKZtRzNlJoBrQL1M4NiXeo7rZm/k7dWJTIgJ4smxDbfzoWiRipVSUzC28v7B9JxjA8YjALrfADd9Aml74KPRdZ7O0crRnvdm9MXF2YG7Pt1OVl6RlQIVomUxJ4Gep5TyBp4GvgPigJdtGpUQ1Sgt07y9OhFHe0VOYQlxJ+reH1ilzAM2738uLi3jjZUHmTDnN87kFvHhrX15dVJP3Jxl3LqoV7cBg4D/aK2PKqXCgAUNHJMA6HodzFhiLCz8aDRk1G0T4LaerZg7vQ8nzxXw0Be7KCmVCYZCWKraBFopZQdka63Paq3Xm3anCtBav2fOyZVSY5RSB5RSiUqpJyv5/EylVKZSKtb0uLOWfw7Rgizfm0Zixnn+NsZoq9h85LRtLlQPEzjiTmQzfvZvvLHyENf2bMeKRy5nZFQbm11PiKporeO01rO01l+YiibuWuuXGjouYRI6BG5bbsyJ/mgMJG2p0+n6dPDm39d3Z8OhU7LVtxC1UG0CbRqs/2BtTqyUsgfmAGOBrsAUpVTXSg79UmsdY3p8UJtriZajrEzz9upDRAS4cdtlYYT5ubLlqI0S6OwTUJhtzGa1suLSMt5edYjxczaSkVPIvBl9eH1yDF4uTla/lhDmUEqtVUp5KKV8gN3Ax0qp1xo6LlFB22i4Y4XRUvbpeDjwc51ON6lvCDMHh/LBxqN8syPFSkEK0TKY08Lxq1LqcaVUiFLKp/xhxuv6A4la6yNa6yJgETC+TtGKFu+nfSc5mH6eh0ZEYG+nGBDmw5ajZ2zTB33RAsIKO9rXyYGTOUx8ZxOv/nqQsd0D+fWRy7mqW1urnFuIOvDUWmcDE4GPtdZ9gFENHJO4mHco3L7CaC1bNBV2fV6n0/3fuCgGd/TlqW/3EpucZZ0YhWgBzEmgbwceANYDO0yP7Wa8LgiouNohxfTcxW5QSu1RSn2tlAox47yihSor07y56iAd/V25pkc7AAaG+5JTUEJ8mg36oMtH2PlH8eOeNPr9ZyUZ2QV1OmViRg7Xzt7Iiax83p3Wm7em9MLbVarOolFwUEoFApP4cxGhaIzc/GHmDxA2FJbdDxtfN1rOasHR3o45U3vTxsOZez7bXud7XLmzuUXsSz1nlXMJ0RiZsxNhWCWPcDPOrSo73UUffw+Eaq17ACuBTyo9kVJ3K6W2K6W2Z2ZmmnFp0Rz9vN+oPs8aGYm9nfHPa0C48WaITfqgMxPAxQ9cffl2Vwqnzhcxd92ROp3yzVWJONoplj88lLHRMiFMNCrPAb8Ah7XW25RS4cChBo5JVMXZHaYuNqZ0rHwWfvk/KKvdYkBvVyfmzehLdn4J9yzYQWFJ3efrP/pVLBPm/CZVbdFsmbMT4S2VPcw4dwpQsaIcDJyoeIDW+rTWutD04ftAn8pOpLWep7Xuq7Xu6+/vb8alRXNTVqZ5a9UhwitUnwECPVvTwdeFzUfqvt3tJTISwL8LBcWlbEw8haO94vMtx8nIqV2FJjEjhx/2nOCWwaG08Whl5WCFqBut9WKtdQ+t9X2mj49orW9o6LhENRycYOIH0P8e2DwHvppR610LowI9eG1ST3YlZfHKzwfqFNaxU7msOZBJqdbM+mIXOQXFNb9IiCbGnBaOfhUeQ4FngevMeN02IFIpFaaUcgJuxhiD9wfT24XlrgPqNptHNFsr4k6ScDLnj97nigaG+bLt2BnKrNkHXT6BI6ALvx8+TUFxGf+4pislZZp5taxCv706kdaO9tw11Jw3cISoX0qpYKXUt0qpDKVUulLqG6VUcEPHJWpgZwdjX4bRL8KBn+D9EX+2n1lobHQgN/UJZsHm45w+X1jzC6rw6e/HcbBTvDO1N6lZ+fxj6b5an0uIxsqcFo6HKjzuAnoBNTZtmna0ehDjLcF44Cut9X6l1HNKqfIEfJZSar9SajcwC5hZ2z+IaL6M3udEwvxcubZC9bncgHAfzuUXE3/Sin3QOWlQeA78u7AqIR0XJ3sm9Q1hfEw7Fmw5TmaOZd9cEjPO893uE8wY1AEf6XkWjdPHGEWOdhjrVb43PScaO6Vg0P1w63dQkGUk0XHLanWqe64Ip7CkjM82H6/V63MLS1i8I5mx0YGMjQ7k4ZGRLI09wZKdMuVDNC/mVKAvlgdEmnOg1nq51rqT1rqj1vo/puee0Vp/Z/r9U1rrblrrnlrr4VprGUYpLvFrfDrxadk8ODwCB/tL/8kOCPcFYIs12zgyjX+K2r8zq+MzGBLhRytHex4aEUlRSRnz1h+26HSzVx+ilYM9d0v1WTRe/lrrj7XWJabHfEB65pqS0CFw9zpjctBXt8Cv/4Qyy/qZIwLcGRUVwKe/H6eg2PJe6KWxqeQUlDBzcAcAHhgeQf8wH/6xdB/HTuVafD4hGitzeqC/V0p9Z3r8ABwAavejrRAW0lrz5spDhPq6MD7m0uozQJBXa0J8Wlt3IaHpLdCDZcGcOFfAKNPmJmF+rkyICeKzzcc5ZeZbnEcyjerzLYM64OvmbL0YhbCuU0qp6Uope9NjOlDjfyql1Eemto9K36dXSk0zTVrao5TapJTqafXIxZ88g4wNV/rcBr+9AQsmQq5l98a7hoZzJreIry2cDa215pNNx+jWzoPe7b0BsLdTvDE5Bgd7O2Yt2kVRiex6KJoHcyrQ/wNeNT1eBC7XWl+yq6AQtvBrXDpxadk8OCKy0upzuYFhvmy1Zh90Rjy09uHXYyUADOvyZyHuwRERFJWU8f5683qhZ69OxNnBnrsul+qzaNRuxxhhdxJIA27E2N67JvOBMdV8/ihwhWna0vPAvLqFKWrk4AzXvgHXvQ3HN8G8YXAi1uyX9w/zoWeIFx9uPGrRjP3NR85wMP08tw4ORak/16q082rNyzf0YE/KOV79tW4LFIVoLMxJoJOALVrrdVrr34DTSqlQm0YlBKbq86pDdPB1YUIV1edyA8N9ycor5kB6jnUunnkAAqJYmZBJz2BPAtz/nJoR7u/GdT3b8envNVehj57KZWlsKtMHtsdPqs+iEdNaJ2mtr9Na+2utA7TWEzA2VanpdeuBKvuntNabtNZnTR9uxpjIJOpD71vg9p9Bl8JHoyF2oVkvU0px99Bwjp7KZWV8utmX+2TTMbxdHLmu56X36zHd2zJtQHveW3eEDYdkHK1o+sxJoBcDFd9zKTU9J4RNrYrPYP+JbB6oove5IqvOg9YaMuPJ94pgd0oWI03tGxU9OCKSgpJS3t9QfRV69upEnBzsuPvyjnWPS4j696iVz3cH8JOVzymqE9TH6IsO7gdL74NlD0JhzYWG0d3aEOLTmnlmvtOWmpXPiriTTO7XnlaO9pUe8/S4rkQGuPHoV7vNboETorEyJ4F2MG3FDYDp9zJGQNhUefU5xKc11/eqbAPLCwV7uxDs3do6CwnPp0PBOeJLgtAaRnQJuOSQiACjCv3Z78c5k1tUyUmMWahLY1OZNqAD/u5SfRZNUmUbYtXuREoNx0ig/1bNMbJpli24+cOMpTDkUYj9HN4dDMd+q/YlDvZ23DkknB3Hz7LjeM331c9NUzumDWhf5TGtnex5e2ovzuUX89fFu9G13D1RiMbAnAQ6s8LYOZRS44FTtgtJCFhzIIO9qed4cHgEjjVUn8sNCPNly9HTde+DNk3gWH/Wh7YerejWzqPSwx4aEUF+cdVV6NlrEnGwU9xzhfQ+iybLKhmOUqoH8AEwXmtd5dtEsmmWDdk7wKh/wm0/gbKD+eOM3QuLq94Y6qa+wXi2duT99UerPXVBcSmLtiUzMqoNIT4u1R7bpa0HT4+LYs2BTD7+7Vht/iRCNArmZCb3An9XSiUppZIwqgf32DYs0dK9vTqRYO/WTOxtfrvkwHAfzuYVcyjjfN0unmEk0N+muDMiKuCCxTAVRQS4c02Pdnyy6dglVejjp3P5dlcqUwe0v6B/WojGRimVo5TKruSRgzETuq7nbw8sAWZorQ/WOWBRN+0Hwr2/Qd/b4PfZ1S4wdHFyYMbADvwSd5Kj1Yyg+3FPGmdyi5g5ONSsEGYM7MCoqDa89FMC+1LP1eIPIUTDM2cjlcNa64FAV6Cb1nqw1jrR9qGJlup8YQm7krK4qU+I2dVnMBYSghX6oDMTKHby4niRGyMrad+oaJapCv3BRVXoOabq831XSO+zaNy01u5aa49KHu5aa4eaXq+U+gL4HeislEpRSt2hlLpXKXWv6ZBnAF/gHaVUrFJquw3/OMIczm5wzesw7WvIPwsfjIR1/4XSkksOvXVwKI52dny4sfJ32rTWfPL7MSIC3Bjc0desyyuleOXGHni7OjLri12cy5OtvkXTY84c6BeUUl5a6/Na6xyllLdS6t/1EZxomRLSjB0FuwdV3jpRlRAfF4K8rDAPOjOBE44dcHawZ3BHv2oPjWzjztXRgXyy6RhnTVXo5DN5LNmZypT+7QnwkOqzaN601lO01oFaa0etdbDW+kOt9Vyt9VzT5+/UWntrrWNMj74NHbMwibwS7v8duo6HNf82JnWcurA+5u/uzMTeQSzenlLp9t6xyVnsSTnHrYM6VPluXWV8XJ146+ZepJzN57b5W8krujR5F6IxM6e8N1ZrnVX+gWkc0dW2C0m0dHGmBDoq0LIEGoxpHFuOnqn94hSt0Rnx7Cpsw5AIP1o7Vb6avKJZIyLJKy7lw41Gn+CcNYnY2SnuGybVZyFEI+fiAzd+BDd8CKcTYe4Q2DwXyv4cvnXn0LAqt/f+ZNMx3JwduN6CdrtyA8J9eWtKDLHJWdzz2Q4KSyzf+VCIhmJOAm2vlPpjhIBSqjUgIwWEzcSnZePl4kigp+XV24FhvpzJLap9H3RuJqogi9j8toyIqr59o1zntu5c3T2Q+ZuOsS/1HF/vSGFKvxDaSPVZCNFURN8I9282tgP/+W8w/+o/qtFVbe+dmVPIj3vTuLFPMG7ONXb7VGpM90BemtiDDYdO8eiXuy3auEWIhmROAr0AWGXqa7sd+BX41LZhiZYs7kQ2XQM9LHo7sFx5H/SW2rZxZMQDcEgHVTq+rioPjYzgfGEJ0z/cgp1S3CvVZyFEU+MRCNMWw4R3ISMO5l4Gv70JpSV/bO/9zc4/t/f+YmsSxaWaGYM61Omyk/qF8PS4KH7cm8bTS/dabbxdenYBK+PSee3Xg9w+fxsj/reWL7YmWeXcQtT4I6PW+hWl1B5gFMZM0Oe11r/YPDLR5JzLK+b1lQe5f1jHWvf+lpSWkXAyh+kDa3dDDvFpTaBnKzYfOcOMQaGWnyDTtM2sfxSBnq3NflmXth6M7d6Wn/adZMbADha9VgghGg2lIGYqdBwBPz4Gvz4D+5fSf/xsegZ78sGGo9zcrz1lWvP5luMMjfSjo79bnS9759BwsvKKmb0mEc/WTjw5totFr8/MKWRPitGPvS/1HHtSz5GZY/Rs2yljdn9rJ3ueWrKXkjLNjFp+jxGinFnvuWitfwZ+BlBKXaaUmqO1fsCmkYkmRWvNX7/ezYq4dEJ9XZh5WVitznPsdC6FJWV0rUX/MxiruweG+7LhUCZaa4ur2IUn9lOgXejTtbPF1358dGdyCkp4YHiExa8VQohGxb0tTF4A+7+F5Y+j3ruCVzrfx7hdfVkZn05JqSY9u5AXro+22iUfu6oTWflFzF13GM/WjjWuI9Fas+3YWeatP8KqhHS0NvL/jv5uDI3wo3uQJz2CPenazgMXJwcKS0p54POd/GPpPgBJokWdmJVAK6VigCnAZOAoxkxPIf4wf9MxVsSlo5SxKru29p8wFhB2rWLzEnMMDPfh212pHM48T0SAu0WvzUnZx1EdzIiubS2+bkd/NxbcOcDi1wkhRKOkFHSfCGGXw09/o/O+t/ipdSjvrXqMJKdIQnxaM6yz+a1uNV9O8dx13cnOL+HlnxPwcnFkSv9LdzYsKS3j5/0neX/9EXannMPbxZEHhkVweSd/urXzwLWKfmxnB3vmTOv9ZxKtde3eqRSCahJopVQn4GaMxPk08CWgtNbD6yk20UTsScniheXxjOwSgJ2dYndK7Qfjx6Vl42RvV6e3BAeEGX3Qvx85Y3EC7Zx1iGT7fkwI8qz19YUQollx9YMbP4TuEwla8jAvnX6YD0rH0WrEE9jbWW23dwDs7BSvTupJTkExf/92Lx6tHBnXIxAw9gj4alsyH/12lJSz+YT5ufLvCd25oXewWROT4KIketl+AEmiRa1UV4FOADYA15ZvnKKUeqReohJNRnZBMQ8u3IW/mzP/u6knX2xL4te4dLLyivBycbL4fPFpOUQEuOHkYP4GKhfr4OtCW49WbDly2qK36Iqz03EvPYdD2y7YWfmbghBCNHldxsEDA/j+9Tu51+F7yrZvBs+nodcMsDMvgTWHo70d70zrwy0fbeEvX+6ipKyM+LQcFm45TnZBCf1CvfnHNV0ZFdWmVgm8JNHCGqrLUm4ATgJrlFLvK6VGYiwiFAIw+s+e+mYvqVn5vD21F96uTsQEewHUugoddyK7Tu0bYLwNOCDch81HLJsHfXCvsUFaYGTvOl1fCCGaKxdPP1rd8C6rhi7Czi8Cvn8Y5g6Fw2usep3WTvZ8cGs/IgPceXhRLPPWH2ZopD/f3j+YxfcOZnS3tnWqfpcn0aOiAvjHsv18+vsxq8UuWoYqE2it9bda68lAF2At8AjQRin1rlLqqnqKTzRiC7Yk8ePeNB6/qjN9OvgAEB3safRBJ1neB52RU8Cp84W1XkBY0cBwX06dL+TIqVyzX5N8YAcA3Xr2q/P1hRCiuRobHcjIkWPhtp/gpk+g6Dx8NgEWTobMg1a7jmdrRz69oz9Pje3Cur8OZ8603vRq72218zs72PPOtD6MigrgGUmihYVqfJ9ca52rtf5ca30NEAzEAk/aPDLRqO0/cY7nf4jjik7+3HN5+B/Pu7dyJMLfjd0plifQcSdqvwPhxcrnQVuyrXdBWhy5yhUX35A6X18IIZo9paDbBHhgK1z5HBzfBO8OguVPQN4Zq1zCz82Ze67oSIiPi1XOdzEnBztJokWtWNRoqrU+o7V+T2s9wlYBicbvfGEJDy7chbeLI69N6nlJv3BMiBexyVkWD8OPT8sBsEoFOtTXhQB3ZzYfMe8mfiTzPG0KjpPrEWF8UxBCCGEex1Zw2cPw0E7ofQtsex/e6gWb34XSkoaOrkYXJ9Gvrjgg24qLGtV+pZZokbTW/N+3ezl+Ope3bu6Fr9ulu7r3DPHiTG4RKWfzLTp3XFo2QV6t8XRxrHOc5fOgtxw5bVYivzohg0i7FFyCutX52kII0SK5+cM1r8N9myCoN/z8JMy7ApK2NHRkNSpPoif2CuLt1YmMe2sj249Zp4oumidJoIVFvtyWzLLYEzwyqhMDTG0SF4sJMRYS7rJwHnTciXNWad8oNyDch4ycQo6a0Qe9bf9B/FQ2biHdrXZ9IYRokQKiYPoSYyOW/LPw0VWw7EHINb+lriE4Odjx2uQYPp7Zj/yiUm6c+ztPL91LTkFxQ4cmGiFJoIXZEk5m88/v9jMkwo/7q9ltr3Nbd5wd7NhtQQKdX1TK0VO5dZ7AUVF5H/SWo9VXEbILislJNnamwt/yHQiFEEJcRCmIutbojx48C3Z/AbP7wI5PoKysoaOr1vAuAax45HJuuyyUz7ckceVr61mx/2RDhyUaGbN2IhQir6iEBz7fiUdrR16fHFPt+CBHezuigzwt2pHwQHoOZdo6/c/lwv1c8XNz5psdKRQWV93Pdjgzl3BSjA/8o6x2fSGEaPGc3eCq56HnFPjxMfh+FuxaANe8Bm2ttw24tbk6O/DPa7sxPiaIJ7/Zw92f7eDq6LY8e203AjxaNXR4ohGQBFqY5bvYExzOzOXT2/vj735p3/PFeoZ4sWDzcYpLy3C0r/mNjvIJHN2sWIFWSnFl1zZ8sTWJ7cfPVnvs/1zT0fbuKI92Vru+EEIIkzZd4bblsHsRrHga3rscBtwLw56CVta771tbTIgX3z80hHnrj/DmqkNsOHSKv18dxeS+IbLhVgsnCbQwy8r4DIK8WjM00s+s42NCvPhw41EOnMyhuxnbYsenZePu7ECwd+u6hnqBF67vzhOja27L8PzqXVRJZ5nAIYQQtqIUxEyBTqNh9fPGlI69i6HfXdD3dmMRYiPkaG/HA8MjuDo6kKeW7OGpJXvZfuwsr07q2dChiQYkPdCiRgXFpfyWeIqRUQEoMxPM8oWE5rZxxKVlExXoYfb5zaWUwtvVqcaH3akDENDFqtcWQghRCRcfY1rHnasgMAbWvgCvd4OlD8DJvQ0dXZXC/Fz54q6B3D+sI9/sTOGHPScaOiTRgGyaQCulxiilDiilEpVSVW6+opS6USmllVJ9bRmPqJ3fj5wmv7iUkVFtzH5NsHdrfF2dzEqgy8o08Wl138K71vLOQG4G+EsCLYQQ9Sa4D0z/Gh7YBr2mw/4lMHcIzL8GEn6EssY3i1kpxaNXdqJniBdPL91HRnZBQ4ckGojNEmillD0wBxgLdAWmKKW6VnKcOzALaPyDIluoVfHpuDjZMyDMx+zXKKXoGeJl1iSOpDN55BWVEhXoXpcway8zwfhVFhAKYTGl1EdKqQyl1L4qPq+UUm+ZCil7lFK96ztG0cj5dzIWFT6yH0b9C84chUVT4e3eRptHQXZDR3gBB3s7Xr2pJ/lFpTy5ZK/Fm4aJ5sGWFej+QKLW+ojWughYBIyv5LjngVcA+TGuEdJaszo+gyERfrRytLfotTEhXiRmnq9xhmZcmnFz7BpYc6+0xXLSIWF59Y/9S41jZYSdELUxHxhTzefHApGmx93Au/UQk2iKXHxgyF/g4d1w03xwa2NsxvJaV/j5KSOxbiQiAtx4cmwXVidk8NX25IYORzQAWy4iDAIq/qtKAQZUPEAp1QsI0Vr/oJR6vKoTKaXuxrjx0r59exuEKqqScDKHE+cK+MuoTha/tmeIF1rD3pRzDI6oevFh3Ils7O0UkW3c6hLqpbSGRVMgdUfNx7r4gWewda8vRAugtV6vlAqt5pDxwKfaKNNtVkp5KaUCtdZp9RKgaHrsHaDb9cYjdYdRhd46z/i1yzgYeD90GNzgi75vHRTKiv3pPPd9HIM7+hHi49Kg8Yj6ZcsEurJ/2X+8z6GUsgNeB2bWdCKt9TxgHkDfvn3lvZJ6tCo+HYBhXSxfHR0T/OeOhNUl0PFp2UT4u1lc4a5R4krj5jvyGeg4svpj3QMb/GYsRDNVWTElCLgkgZZiibhEUB+44QO48jnY9gFs/wgSfoC2PYxEuvtEcKh5tKot2Nkp/ntTD8a8sYHHF+/mi7sGymi7FsSWLRwpQEiFj4OBiktW3YHuwFql1DFgIPCdLCRsXFYlZNAz2JMAd8sHx3u6OBLu51pjH7QxgcPK/c9aw9oXwbM9DHoI2sVU/3A3f4GkEMKvGg/lAAAgAElEQVQi1RZTLnhS63la675a677+/o1zpJloIB7tjGLII3Fw7ZtQWgRL74U3omHdK3A+s0HCCvZ24Zlru7Ll6Bk+3nSsQWIQDcOWCfQ2IFIpFaaUcgJuBr4r/6TW+pzW2k9rHaq1DgU2A9dprbfbMCZhgVPnC4lNzmJEl9onlz1DvIhNzqpykcWZ3CLSzhVYfwJH4iqj+jz0UXBwsu65hRCWqKmYIoT5nFygz0y4fzNMX2JUotf8xxiD9/UdcGRtvW8VflOfYEZFBfDKzwkkZpw3+3Vaa8rK5E31pspmCbTWugR4EPgFiAe+0lrvV0o9p5S6zlbXFdazJiEDrWFkVECtzxET4kVGTiEnqxj1E2+LBYRaw7qXwDMEYqZZ77xCiNr4DrjFNI1jIHBO+p9FnSkFESP/HIPX+xZI/BU+HQ9vxRhV6XMp9RSK4oWJ0bg42fPYV7GUlFafwJeUlrF0VypXvr6e6Gd/4cWf4snMKayXWOsi+Uwe//0lgaGvrOa+BTtIb+Ej/Gy6E6HWejmw/KLnnqni2GG2jEVYbnVCBm08nOu0vXbP8g1VkrIIjL50l8HyBNqqLRyHV0PKNmNQv1SfhbAppdQXwDDATymVAvwTcATQWs/F+B5wNZAI5AG3NUykotny7wTj/gdXPQ/xP8CuT42q9JoXjCS71wzofLVNvx8EuLfiP9dHc//nO3ln7WFmjYy85Jji0jK+3ZXKO2sSOXY6j05t3Liisz/vrz/C/N+OMblfCPdc0ZEgL+vuyFsXJaVlrErIYOGWJNYfykQBA8N9WZ2QwcbElr2tuWzlLSpVVFLG+oOZXBcTVKfdAaMC3XGytyM2JYux0YGXfD7uRDZtPJzxdbPSIhCtYd3L4BEMMdOtc04hRJW01lNq+LwGHqincERL5tgaetxkPM4chdiFEPs5LL4VXHyhx83Q51abjSy9OjqQ8THteGvVIYZ3DiA62HhntbCklMXbU3h37WFSs/Lp1s6DudP7cFXXNtjZKY6eymXu2sN8sTWJhVuSuL5XEPcO60hHfytPprJAalY+X25N4svtyaRnF9LGw5mHRkRyc78Q2nm15tipXJ5aspenluxlWWwqL07sQZifa4PF2xAkgRaV2nr0DLlFpYyqQ/sGgLODPVHtPIhNqnwhYVxaNl0Drdj/fGQNJG+Bca9K9VkIIVoqnzAY8X8w7Ek4vMaoSm+dB5vnQIch0Pc2iLrW6hM8nruuO5uPnObRr2L5+t7BfLMzhffWHyY9u5Be7b3494TuDOvsf0FhKszPlZdv7MHDoyKZt/4Ii7Yl8fXOFK6ODuT+YR3p1s4GeyRUYf3BTOZvOsbaAxlo4IpO/jw/vj0jugTgYP9n12+onysL7xrAl9uS+c/yeMa8sZ6/jOrEnUPDcLS36SbXjYYk0KJSK+PTcXawY3DHqsfPmatXiBdfbU+mtExjX+FtnsKSUhIzzjOiS92S9D9oDWtfBo8g4y07IYQQLZudPUSOMh7nM42K9I6P4Zs7jPn/vaYbixJ9wqxyOU8XR16+oQczP95GvxdWUlRSxoAwH16bFMPgjr7VvqPbzqs1z17XjQdHRPDRxqN89vtxftyTxrjoQN68OeaCBNYWft6Xxr0LduLv7sz9wyKY3C+k2tnWSilu7m8k188s28/LPyfww54TvHxDD7oH1V/S31Baxo8JwiJaa1YlpHNZhB+tneo+m7lniCd5RaUcysi54PlD6ecpKdPWm8BxdB0kb4YhjzTYXFAhhBCNlJu/sdPhQ7tg+jfQfiBsettYdPjZRKN/urSkzpcZ1jmAWSMiGBrhx1f3DOLLewZxWYSf2e2Qfm7OPDGmCxufHMHdl4fz4940Vpr2ZLCVI5nneXzxHnqGeLHhieE8Prqz2RvDBHi0Yu6MPsyd3puMnELGz/mNF3+KJ7+o1KYxNzSpQItLHM48T/KZfO69oqNVzhcT4g3A7uQsurT9M1n+cwtvKyTQ5dVn93bGamwhhBCiMnZ2EDHKeJxLhV2fwY5P4MtpxveQ7hONRYchA4xdEWvh0avq3mft2dqRJ0Z35sc9aczfdIwx3S9dR2QNeUUl3LdgJ472inem9a71pmZjugcyKNyPF5bH8966I5w8V8CbN/eycrSNh1SgWwBL50yujM8AsFprRaivC56tHYm9aEOV+LRsXJzs6eBrhYUHR9dD0iapPgshhDCfZ5DRJ/2XvXDzQmjbHba8B/Ovhv9FwJJ7YP9SKMyp+Vw24GBvx4xBHdh85AwJJ7Otfn6tNX9fspeDGTm8NaVXnSeAeLo48vKNPZgxsAM/7ztJbmHdK/qNlSTQzdycNYkMfWUNp86bP2NydXwGXQM9CPS0zigdpZRpQ5VzFzwfdyKbzm3dL+iLrrV1LxvbcUv1WQghhKXsHaDLOJi2GJ44Ajd9ApGj4dAvxhSPV8Lhs+th6/uQlVzz+axoct8QnB3s+MQGOx0u2HycpbEneHRUJ4ZGWm/3z2t7tqOwxBiB11xJAt3MLd2VSmpWPo98GWtWJTorr4jtx8/UafOUysQEe3LgZDZ5RcZPo1pr603gOLoBjv9mVJ8dLd9yXAghhPhDKw/oNgEmvgePJ8JtP8GAeyArCZY/Dm90h3nDjWQ674zNw/F2deL6XkF8uyuVrLwiq513V9JZnvshjhFdAnhgeITVzgvQp4M3/u7OLN/TfPdMkgS6GUs6ncehjPP0bu/FhkOneHfd4Rpfs/ZAJmUaRkbVfvvuysS096JMw75U4y2o1Kx8cgpKrLOAcN3L4NYWet9a93MJIYQQ5ewdoMNguOrf8NAOeHA7XPkclBYbyfT/OsGiaZDwI5RYL7m92K2DQykoLuPLbdapfp8+X8j9n++kjUcrXp8UY/WNUOztFGO7t2XNgYxm28YhCXQztirBWLX72qQYrukRyGu/HmTbsep/Wl6VkIGfmxM9rDyCpmewaUfC5LOA0b4BEFXXCvSxjXBsg7GyWqrPQgghbMkvEi57GO7bCPduhP53G3sPLJoKr3WBn/4GJ3YZC9utKCrQgwFhPny2+TilFq5rulhpmebhRbGczi1i7vQ+eLo4WinKC10dHUhhSRmrm2kbhyTQzdjqhAw6+rsS6ufKixOjCfFuzUMLd3Emt/KfkotLy1h3IIPhnQOs/tOor5szIT6t/1hIGJeWjVLQpW0dt/Be+xK4tTHmeAohhBD1pW00jHkBHk2AqV9B6FDY/hHMGwbvDIINr8KpRKtdbubgUFLO5rOqjiPt3lh5kI2Jp3h+fDebzmvuF+qDn5szy/c2zzYOSaCbqZyCYjYfOf1HK4Z7K0dmT+3NmdwiHl+8u9J+6O3HzpJdUGL19o1yPYO92G1aSBh3IpswP1dcnOowSfH4JqP6fNnDxhauQgghRH2zd4BOo2HSJ/D4QbjmdXB2h1XPwew+MGcgrP4PpO2pU2X6yq5taOfZivl1WEy4Kj6dt1cnMrlvCJP7ta/1ecxRsY2jfP1TcyJzoJupjYdOUVyqGVlhFF33IE+eviaKZ5bt54ONR7j78gvnPK9OSMfJ3o4hkXXffbAyMSFe/LAnjYycAuJPZv/R1nGJslL45FpI31f9CYsLwDUA+txm/WCFEEIIS7X2hr63G49zKUZvdPz3sOF/sP4V8OpgbCEedR0E9zNmUpvJwd6O6YM68MrPBziYnkOnNpa9g5t0Oo9HvoylWzsP/jW+m6V/slq5OjqQzzYfZ3VCBtf0aFcv16wvkkA3U6sSMvBo5UCfDt4XPD9jYAc2HznNKz8foG+oD73be1/wmgHhPrg52+afRUyIkTBvOHiK5DP53FzVT7/7vjGmakTfBC6+1Z+00xhwMm+3JCGEEKLeeAYb0zsG3GNsI35guZFMb3kPfp9tLH7vdBUE9zeSab9ONSbUN/drzxsrD/HJpmP85/pos0PJLyrlvs93APDutD613izFUv3D/mzjqE0C/eOeNJbFpnLfsI70au9d8wvqkSTQzVBpmWZNQgbDOgfgYH/hf0alFC9O7MHe1A08tHAXP84agpeLE0dP5XIkM5dbBnawWVzdgzyxt1N/rCKudAJHWSmsewUCusH18yz66VwIIYRolNz8oc+txqPgHBxcAfHLIG4Z7PzUOMbZA4J6G8l0UF8I7guuF74j7OPqxPie7ViyM5UnRncxawFgSWkZD32xk7i0bD68tS/tfeuv6GRvpxjTvQ1f70ghr6jEorbNguJSnv1+P5k5hayIS2dMt7Y8ProzEQFuNozYfJKdNEO7U7I4nVtU5Sxnz9aOzJ7Sm4ycAh5fvAet9R+LEmzV/wzQytGeLm3d2WqaBNKtsgkc+7+F04fgiickeRZCCNH8tPKEHjfB5AXwxDFjNN6EdyH6RmOu9IbX4IvJ8N+O8GZP+OZOY+b0yb1QVsqtg0PJLy5l8Y6aR9pprfnHsn2sjM/gueu6MaKL7b7HV+Xq6EAKistYk5Bp0esWbkkiM6eQD2/tyyOjOrHhUCaj31jPk9/s4eS5AhtFaz6pQDdDq+MzsLdTXNGp6l2FeoZ48eTYKJ7/IY6PfjvG6oQMOrVxI8THtj+ZxoR4sf9ENr6uTvi7X7TldlmpMdM5oKvRHyaEEEI0Z3Z2xmg8v0iImWo8V5QLJ2IhdTukbIOj62HvYuNzzh50D+7Ly37t+H3jIUr73YF9q6p7od9cdYgvtibz4PAIZgwKtf2fpxIDwnzxc3Ni+b40xvUINOs1BcWlvLvuMAPDfRgZ1YaRUW2YNrA9s1cn8vmW43y7K5XbLgvjvis62mwMX00kgW6GViVk0KeDN14uTtUed/tloWw+cpqXfopHa7hzaLjNY+sZ4sXnW5Lo2s4DpS4albf/Wzh1EG78WKrPQgghWiYnVwi9zHiAMbkj6zgkbYGk3yF5C5POr2EymrKXn4O23aH9IAi7AkKHGDspYlRw31h5iJv6BPPYVZ0a7I9jb6cY3a0tS3amkl9USmunmvuvPzdVn9+e0uuP5/zcnHn2um7cMSSM1349yHvrD7Nwy3HuHx7BzMGh9dbXXa7FZylaa77ffYKZH28lNSvf5tdLOp3HA5/vtNlcxNSsfOLTshllxlbcSin+e2MPAtxbUVKmzXpNXfUyLSS8ZAvvsjJY/1/w7wJdJ9g8DiGEEKJJUAq8Q6HnZLj2Dbj/d0r+epRHHJ7me/dJRu/0jk9g0RR4ORQ+vIrDX/2dpcsWM6qTNy9MjL60YFXPxkUHkl9cytoDNW+qUlBcytx1hxkU7svA8EsHCYT4uPD65Bh+fGgovTt489JPCVz1+nqyC4ptEXqVWnQF+tT5Qv6xdB8/7TsJwKwvdrHo7oE42lv/54qyMs2CLcd56acE8opK2XbsDCO6BFj9J6byHX/M7XPycnHivRl9WLIztV5WuHb0d+O+YR25oXfQhZ+IWwqZCXDjR1J9FkIIIarh6OpNx8ETeHjFQbpNv5wIHydI3gpH1pIb/yuhSe/wlZNGn3RFLRoCHYcbFWr/Lg3yPbZ/mA++rk78uDeNsdHVt3Es2HyczJxCZleoPlemazsP5t/Wn9UJ6dw+fzufbjrGgyMirRl2tVpsprJ8bxpXvb6eVfEZPDGmM29MjmHH8bO8uuKg1a+VfCaPaR9s4Zll++nTwZvXJ/ckI6eQRVuTrH6t1fHpdPB1oaO/q9mv6R7kyTPXdsXeyrsPVsbOTvG3MV2ICKjQs1VWZkze8Oss1Wchmhil1Bil1AGlVKJS6slKPt9eKbVGKbVLKbVHKXV1Q8QpRHMzpX97nBzs+GTTcXBwhrChJEY/wmVnnmaC2wJyxn+M6nEznE6En5+EdwfBi8Hw4VXw42NG1Tp1p7Gngo052NsxuntbVidkUFBcWuVx+UWlzF13hMEdfRlQSfW5MiO6tGFklwA+2HiU84X1t2FLi6tAn8kt4pll+/hhTxrRQZ68OqnnH8PItxw9w9x1hxkQ7sPwznVvZ9Bas3BrEi/8GG8aHxfNzf1CUErxxdZk3l13mJv7t7daFTqvqITfDp9m+oAODf52jUXiv4PMeLjhQ7Cr3x4mIUTtKaXsgTnAlUAKsE0p9Z3WOq7CYU8DX2mt31VKdQWWA6H1HqwQzYyvmzPX9mjHNztT+OuYzuQXlXLrR1txsLPjnTtG4O7jAr0mGgdnJcGxjZC229gRcfeXsO0D43N2DkYBq200BPYw+qjbRFu9Uj0uOpCFW5JYeyCDMd0rr0J/vuU4p84XMmdq9dXni80aGcn4Ob/x6e/HuH9YhBWirVmLSqB/3neSp5fu5Vx+MX8d3Zl7Lg+/YE7yP6/tyq6kszz21W6WzxpKW89Wtb5WalY+T36zhw2HTnFZhC8v39CDYO8/J1z8ZWQkUz/Ywlfbk7nFSitjf0s8TVFJWZXj6xqlsjJj8oZfJ+h2fUNHI4SwTH8gUWt9BEAptQgYD1RMoDVQvujBEzhRrxEK0YzNHBzKNztT+GjjUX7ed5Jz+cUsunvgpRO1vNobUz7KJ32UlUHWMSOZPrkXTu6BI2thzyLj864BEDEKIkZCxxHg4lPnWAeE+eDj6sSPe09WmkAb1efDFlWfy/UM8WJYZ38+2HCUWweF4mqjDeEqahEJ9NncIp79fj/LYk/QrZ0HC+4cQJe2l84gbuVoz5xpvbn27Y3M+mIXC+8acMlGJDXRWvPV9mSe/yGeMq3594TuTBvQ/pKK8KCOvvQL9eadNYeZ3C8EZ4e6V15XJ6Tj7uxAv9C6/0OvNwnfQ0YcTHxfqs9CND1BQMVhtCnAgIuOeRZYoZR6CHAFRtVPaEI0f9HBnvTp4M0bKw/haK/4eGZ/ugd51vxCOzvwCTce3Sq0TmanGYl04q9w8CfYvRCUHQT1MSXUV0K7mFp9v3awt2N0t7Ysi02loLj0knffjepzEe9Mq93EkFkjI5n4ziYWbD7OPVd0rNU5LNHse6DXHsjgqjfW8+OeNB4Z1YmlD1xWafJcrqO/G/+5vjtbj53hzVWHLLpW2rl8Zn68jb99s5fuQR788pfLmT6w8nYKpRQPj+zEyewCvtpW8zD0mpSVaVbFZ3B5J3+cHJrIX2t577NvBHS/oaGjEUJYrrJeMX3Rx1OA+VrrYOBq4DOlVKU3KaXU3Uqp7Uqp7ZmZlm26IERLdc/l4TjYKf53U0+GRPrV/ILqeARCzBRjQf9fD8Odq+DyJ4xRemtfgg9GwH8j4PNJsPJfsPdryIiHUvMmYIyLDiSvqJS1By78/51XVMLcdYe5LMKX/mG1KwL2bu/N0Eg/5q0/Qn5R1X3W1tLsK9BFJWX4uTkz/7Z+dGtnxk9lwPW9gtmUeJrZaxIZEOZb4z9IrTWLd6Tw/A9xlJRqnr22K7cMCsWuhkV5l0X40reDN++sPcykOlah95/IJiOnkBFdmlD7xoEfIX2factuqT4L0QSlACEVPg7m0haNO4AxAFrr35VSrQA/4JJ5VlrrecA8gL59+16ciAshKnFVt7bsfXa0WfOVLWJnb2wnHtwXhj9l7JJ4eDUkroK0WDi8CspMi/bsnYw+6jZdoU03COgG/p3AI+iC7+8Dw402juV70xjTve0fz3++OYlT54t4d1Td5lU/PDKSG+f+zudbjtt8b4tmn0Bf1a0tI6PaWDxh4l/juxGbnMVfvtzF8llDCfCovB86PbuAJ7/Zw5oDmfQP9eG/N/Wgg695EzCUUjw8KpIZH25l8fYUpg/sYFGMFa1KSEcpGNa56t0HGxWtjd5nn45SfRai6doGRCqlwoBU4GZg6kXHJAEjgflKqSigFSDlZSGsyOrJc2VcfIztxqNvND4uKTI2P8uIM4ph6XFwdAPs+fLP19g7gVcH8AkD7zAcfMKZFQJfxydRkN+ZVq1dyCsq4b31hxkS4VfnFtS+oT5cFuHL3HVHmD6wg003V2n2CTRQq/FsLk4OzJnWm+tmb+ThRbEsuHPABefRWvPtrlSe/W4/RaVlPHNNV2YOrrnqfLEhEX70bu/FO2sSmdQ3pNbtF6sTMugV4oWvm3PNBzcGB5YbCxcmzAX7FvHPUIhmR2tdopR6EPgFsAc+0lrvV0o9B2zXWn8HPAa8r5R6BKO9Y6bWWqrLQjR1Dk7GLohtuwOT/nw+74zR1nH6EJw5CmeOwNmjcPx3KMphJjDTDvTLj4FnMGdVGx4rcGdIQD/YfwK8w4yEu5V5XQMXmzUiksnzNrNwSxK3Dwmzxp+0UjbNXJRSY4A3MW6sH2itX7ro8/cCDwClwHng7ovGHzWoTm3ceW58d574eg+zVyfy8ChjQHdGdgF//3YvK+Mz6NvBm//e1JMwP/PnLldkVKE7cetHW1m8I5lpAyyvQqdnF7An5Rx/Hd25VjHUu/JeKp9wiL6poaMRQtSB1no5xmi6is89U+H3ccBl9R2XEKKBuPhcuBV5Oa0h9xQlpw/zzPwfGOqTzZWBeZzat5urnRLx3LkGdlY4vrWPqXIdCp4h4BUCnu1Nv4aAs1ullx8Q7svAcB/mrjvM1AHWGxV8MZsl0GbOB12otZ5rOv464DVMvXKNxU19gtl8+DRvrjpIvzBvMrIL+ed3+ykoLuXpcVHcdllYnTcguTzSj5gQL95Zc5ib+lhehV5j2n2wyYyvO/CTMTJn/DtSfRZCCCFaAqXAzR8HN39Ku7nw171p3NM1nFcLDvL1vYPoG+gIZ48ZVeuzR//8NXUHxH0HZRctVGztbUqs2xsP347gGwm+EcwaEcHUD7ZadVTwxWyZvdQ4H1RrnV3heFcuXb1dd0c3QMKPtX65Al52LeMytxMcnP+ZMZrO3Zmh0X54nV9nvHFZRwp42zuPX9PSObrgGzq3ca/xNRX5xafzimsRnXetp/JF8Y3MoRXGT5Q9Jjd0JEIIIYSoZ1f3COTL7cm8vvIgQyP96Fve+9w22nhcrKwMzp+ErGQ4l2xsDHMu2fj4dKKxsLEk/4/DBzm6stqtLYd/bUtJ3mAc/DsbCXZAV3Cs/R4fFdkygTZnPihKqQeARwEnYERlJ1JK3Q3cDdC+fXvLoshMgNiFlr3mIo7ABKXJty/F0UHhVGaHOmDdRDUYzSTHUvQxjU5zQJmZCGs0/QtKcHSwQ8U2kUkWdnZwzetSfRZCCCFaoMEdffFycSQrr5i/mNpjq2VnBx7tjMelqaSRYOekGX3XpxNRpxLxStpPp9QE7Db8DpQZx9212phpbQW2zGDMmQ+K1noOMEcpNRVjy9dbKzmm9qON+t9lPOrIHqi828Y6FLAtIYPb5m/jpaujubm/eT8orD2QwW0fb2P+1H4Ms8L240IIIYQQtuRob8ddQ8NJOZtHnw5W2PzNzg48g4xH+DAAvLXmznc3cebceVbMDMEp6zD4d6n7tcovabUzXcqc+aAVLQImVPP5Zm9YZ396BHsye00ixaVlZr1mdXwGLk72DLRw20shhBBCiIbywPAIXpzYw2bnLx/ScOxcCd8ku0HUteBUu4EPlbFlAv3HfFCllBPGfNDvKh6glKpYtx8HWLb1XzNj7E4YScrZfL7dmVrj8VprVsWnMyTCz6azDoUQQgghmprLI/3oGeLFHAsKk+ayWQKttS4ByueDxgNflc8HNU3cAHhQKbVfKRWL0Qd9SftGSzOiSwDRQeZVoRNO5nDiXEHTmb4hhBBCCFFPlFL8xYLCpCVsuorLjPmgD9vy+k1ReRX6zk+38+x3+6udL73j+FkAhkvvsxBCCCHEJSq2x17fOwhHe+vUjmUMQiM0MiqA3u29+HxLUo3HDu7oW+U240IIIYQQLZlSilkjIrlnwQ5ik7PqvF14OUmgGyGlFIvvHUxuUUmNx7o6yV+hEEIIIURVRkYFsO6vwwj2drHaOSX7aqTs7RQerRwbOgwhhBBCiCZNKWXV5BlsO4VDCCGEEEKIZkcSaCGEEEIIISwgCbQQQgghhBAWkARaCCGEEEIIC0gCLYQQQgghhAUkgRZCCCGEEMICkkALIYQQQghhAaW1bugYLKKUygSOV3jKDzjVQOHURVOMuynGDBJ3fWqKMUPt4u6gtfa3RTCi2dzrm2LMIHHXp6YYMzTNuGsbc6X3+iaXQF9MKbVda923oeOwVFOMuynGDBJ3fWqKMUPTjbslaYp/R00xZpC461NTjBmaZtzWjllaOIQQQgghhLCAJNBCCCGEEEJYoDkk0PMaOoBaaopxN8WYQeKuT00xZmi6cbckTfHvqCnGDBJ3fWqKMUPTjNuqMTf5HmghhBBCCCHqU3OoQAshhBBCCFFvmnQCrZQao5Q6oJRKVEo92dDxmEMpdUwptVcpFauU2t7Q8VRFKfWRUipDKbWvwnM+SqlflVKHTL96N2SMlaki7meVUqmmr3msUurqhozxYkqpEKXUGqVUvFJqv1LqYdPzjfrrXU3cjfbrrZRqpZTaqpTabYr5X6bnw5RSW0xf6y+VUk4NHaswNMX7PMi93paa4n0emua9vine56F+7vVNtoVDKWUPHASuBFKAbcAUrXVcgwZWA6XUMaCv1rpRz09USl0OnAc+1Vp3Nz33CnBGa/2S6RuZt9b6bw0Z58WqiPtZ4LzW+n8NGVtVlFKBQKDWeqdSyh3YAUwAZtKIv97VxD2JRvr1VkopwFVrfV4p5QhsBB4GHgWWaK0XKaXmAru11u82ZKyi6d7nQe71ttQU7/PQNO/1TfE+D/Vzr2/KFej+QKLW+ojWughYBIxv4JiaDa31euDMRU+PBz4x/f4TjP9EjUoVcTdqWus0rfVO0+9zgHggiEb+9a4m7kZLG86bPnQ0PTQwAvja9Hyj+1q3YHKft7GmeK9vivd5aJr3+qZ4n4f6udc35QQ6CEiu8HEKTeAvFeMvcIVSaodS6u6GDsZCbbTWaWD8pwICGjgeSzyolNpjeuuv0bw9djGlVCjQC9hCE/p6XxQ3NOKvt1LKXikVC2QAvwKHgSytdYnpkKZyL2kJmup9HuRe3xAa7X3nYk3xXt+U7vNg+2Pp2AwAAAQeSURBVHt9U06gVSXPNYV+lMu01r2BscADpreihG29C3QEYoA04NWGDadySik34BvgL1rr7IaOx1yVxN2ov95a61KtdQwQjFHhjKrssPqNSlShqd7nQe719a1R33cqaor3+qZ2nwfb3+ubcgKdAoRU+DgYONFAsZjt/9u7fxC5qjAM48/LbpQlIkEjIkgMYiohgoiIWAQRe1FJgkIQC0mjlSg2gmhhEyRoYzCFEJWARlOJElQURS2Mf4KdLBbGTVIEESRI8lnMGRh0d91BZ+be3ecHw5w5c3c4c5Z5+ebOufdW1S/t/gxwjME/tS+W2nqo4bqoMzMez5pU1VL7IF0CDtHBOW9rtN4GjlTVO6278/O93Lj7MN8AVXUe+Bi4A9iSZL491Yss2SB6mfNg1k9bX3Knj1nf55yHyWV9nwvor4Ed7YjKy4A9wPEZj2lVSTa3Rfgk2QzcC/yw+l91ynFgX2vvA96b4VjWbBhMzX10bM7bwQ6vAT9W1YGRpzo93yuNu8vzneSaJFtaewG4h8Gavo+AB9pmnZvrDax3OQ9m/Sx0OXeG+pj1fcx5mE7W9/YsHADttCkvAXPA4ap6YcZDWlWSGxnsiQCYB97o6piTvAnsArYCS8CzwLvAUWAb8DPwYFV16kCOFca9i8HPTAUsAo8N15t1QZK7gE+B74FLrfsZBuvMOjvfq4x7Lx2d7yQ7GRw4MsdgB8LRqnqufTbfAq4CvgEerqoLsxuphvqW82DWT1ofcx76mfV9zHmYTtb3uoCWJEmSpq3PSzgkSZKkqbOAliRJksZgAS1JkiSNwQJakiRJGoMFtCRJkjQGC2j1VpKLSU6O3J7+H197e5JOnddSkjYis15dNP/vm0id9Ue7TKckaf0y69U57oHWupNkMcmLSb5qt5ta/w1JTiT5rt1va/3XJjmW5Nt2u7O91FySQ0lOJfmgXc1IktQBZr1myQJafbbwt5/1do8891tV3Q68zOAqZrT261W1EzgCHGz9B4FPquoW4FbgVOvfAbxSVTcD54H7J/x+JEn/ZNarc7wSoXorye9VdcUy/YvA3VX1U5JNwK9VdXWSc8B1VfVn6z9dVVuTnAWuH72cZ5LtwIdVtaM9fgrYVFXPT/6dSZKGzHp1kXugtV7VCu2VtlnOhZH2RTxmQJK6xqzXTFhAa73aPXL/RWt/Duxp7YeAz1r7BLAfIMlckiunNUhJ0n9i1msm/JalPltIcnLk8ftVNTy90eVJvmTwJXFv63scOJzkSeAs8EjrfwJ4NcmjDPY+7AdOT3z0kqS1MOvVOa6B1rrT1sXdVlXnZj0WSdJkmPWaJZdwSJIkSWNwD7QkSZI0BvdAS5IkSWOwgJYkSZLGYAEtSZIkjcECWpIkSRqDBbQkSZI0BgtoSZIkaQx/ARR+nSxzt/58AAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "import matplotlib.pyplot as plt\n", - "\n", - "f, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n", - "t = f.suptitle('Basic CNN Performance', fontsize=12)\n", - "f.subplots_adjust(top=0.85, wspace=0.3)\n", - "\n", - "epoch_list = list(range(1,31))\n", - "ax1.plot(epoch_list, history.history['acc'], label='Train Accuracy')\n", - "ax1.plot(epoch_list, history.history['val_acc'], label='Validation Accuracy')\n", - "ax1.set_xticks(np.arange(0, 31, 5))\n", - "ax1.set_ylabel('Accuracy Value')\n", - "ax1.set_xlabel('Epoch')\n", - "ax1.set_title('Accuracy')\n", - "l1 = ax1.legend(loc=\"best\")\n", - "\n", - "ax2.plot(epoch_list, history.history['loss'], label='Train Loss')\n", - "ax2.plot(epoch_list, history.history['val_loss'], label='Validation Loss')\n", - "ax2.set_xticks(np.arange(0, 31, 5))\n", - "ax2.set_ylabel('Loss Value')\n", - "ax2.set_xlabel('Epoch')\n", - "ax2.set_title('Loss')\n", - "l2 = ax2.legend(loc=\"best\")\n" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Model: \"sequential_3\"\n", - "_________________________________________________________________\n", - "Layer (type) Output Shape Param # \n", - "=================================================================\n", - "dense_5 (Dense) (None, 256) 1179904 \n", - "_________________________________________________________________\n", - "dropout_2 (Dropout) (None, 256) 0 \n", - "_________________________________________________________________\n", - "dense_6 (Dense) (None, 3) 771 \n", - "=================================================================\n", - "Total params: 1,180,675\n", - "Trainable params: 1,180,675\n", - "Non-trainable params: 0\n", - "_________________________________________________________________\n" - ] - } - ], - "source": [ - "import keras as ks\n", - "model = ks.models.load_model('train3.h5')\n", - "model.summary()" - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "metadata": {}, - "outputs": [], - "source": [ - "#訓練結果提取，建立混淆矩陣\n", - "import pandas as pd\n", - "import tensorflow as tf\n", - "import sklearn\n", - "from sklearn.metrics import confusion_matrix\n", - "predictions = model.predict_classes(test_features)" - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[1, 0, 0, 1, 0, 2, 1, 2, 2, 1, 0, 2, 0, 0, 0]\n", - "(15,)\n" - ] - } - ], - "source": [ - "from numpy import argmax\n", - "from keras.utils.np_utils import to_categorical\n", - "test_labels_change = [0]*15\n", - "for i in range(12):\n", - " if(np.array_equal(test_labels[i],[0,0,1])):\n", - " test_labels_change[i] = 2\n", - " elif(np.array_equal(test_labels[i],[0,1,0])):\n", - " test_labels_change[i] = 1\n", - " elif(np.array_equal(test_labels[i],[1,0,0])):\n", - " test_labels_change[i] = 0\n", - "\n", - "print(test_labels_change)\n", - "test_labels_change = np.asarray(test_labels_change)\n", - "print(test_labels_change.shape)" - ] - }, - { - "cell_type": "code", - "execution_count": 30, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "(15,)\n", - "(15,)\n" - ] - }, - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
預測值012
實際值
0511
1040
2004
\n", - "
" - ], - "text/plain": [ - "預測值 0 1 2\n", - "實際值 \n", - "0 5 1 1\n", - "1 0 4 0\n", - "2 0 0 4" - ] - }, - "execution_count": 30, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "print(test_labels_change.shape)\n", - "print(predictions.shape)\n", - "pd.crosstab(test_labels_change, predictions, rownames=['實際值'], colnames=['預測值'])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.5" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "#匯入訓練、驗證、測試圖像\n", + "import os,shutil\n", + "\n", + "base_dir = 'Rec'\n", + "if not os.path.isdir(base_dir):\n", + " os.mkdir(base_dir)\n", + " \n", + "train_dir = os.path.join(base_dir, 'train')\n", + "os.mkdir(train_dir)\n", + "validation_dir = os.path.join(base_dir, 'validation')\n", + "os.mkdir(validation_dir)\n", + "test_dir = os.path.join(base_dir, 'test')\n", + "os.mkdir(test_dir)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "#建立訓練集資料夾\n", + "train_b_dir = os.path.join(train_dir, 'be')\n", + "os.mkdir(train_b_dir)\n", + "\n", + "train_h_dir = os.path.join(train_dir, 'ho')\n", + "os.mkdir(train_h_dir)\n", + "\n", + "\n", + "train_y_dir = os.path.join(train_dir, 'yun')\n", + "os.mkdir(train_y_dir)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "#建立驗證集資料夾\n", + "validation_b_dir = os.path.join(validation_dir, 'be')\n", + "os.mkdir(validation_b_dir)\n", + "\n", + "validation_h_dir = os.path.join(validation_dir, 'ho')\n", + "os.mkdir(validation_h_dir)\n", + "\n", + "validation_y_dir = os.path.join(validation_dir, 'yun')\n", + "os.mkdir(validation_y_dir)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "#建立測試集資料夾\n", + "test_b_dir = os.path.join(test_dir, 'be')\n", + "os.mkdir(test_b_dir)\n", + "\n", + "test_h_dir = os.path.join(test_dir, 'ho')\n", + "os.mkdir(test_h_dir)\n", + "\n", + "test_y_dir = os.path.join(test_dir, 'yun')\n", + "os.mkdir(test_y_dir)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "#擷取be資料集的圖片範圍\n", + "original_0dataset_dir = 'be'\n", + "fnames = ['be.{}.png'.format(i) for i in range(20)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(train_b_dir, fname)\n", + " shutil.copyfile(src, dst)\n", + "\n", + "fnames = ['be.{}.png'.format(i) for i in range(45, 54)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(validation_b_dir, fname)\n", + " shutil.copyfile(src, dst)\n", + "\n", + "fnames = ['be.{}.png'.format(i) for i in range(54, 59)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(test_b_dir, fname)\n", + " shutil.copyfile(src, dst)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "#擷取ho資料集的圖片範圍\n", + "original_0dataset_dir = 'ho'\n", + "fnames = ['ho.{}.png'.format(i) for i in range(20)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(train_h_dir, fname)\n", + " shutil.copyfile(src, dst)\n", + "\n", + "fnames = ['ho.{}.png'.format(i) for i in range(45, 54)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(validation_h_dir, fname)\n", + " shutil.copyfile(src, dst)\n", + "\n", + "fnames = ['ho.{}.png'.format(i) for i in range(54, 59)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(test_h_dir, fname)\n", + " shutil.copyfile(src, dst)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "#擷取yun資料集的圖片範圍\n", + "original_0dataset_dir = 'yun'\n", + "fnames = ['yun.{}.png'.format(i) for i in range(20)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(train_y_dir, fname)\n", + " shutil.copyfile(src, dst)\n", + "\n", + "fnames = ['yun.{}.png'.format(i) for i in range(45, 54)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(validation_y_dir, fname)\n", + " shutil.copyfile(src, dst)\n", + "\n", + "fnames = ['yun.{}.png'.format(i) for i in range(54, 59)]\n", + "for fname in fnames:\n", + " src = os.path.join(original_0dataset_dir, fname)\n", + " dst = os.path.join(test_y_dir, fname)\n", + " shutil.copyfile(src, dst)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "12" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "len(test_y_dir)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Model: \"sequential_1\"\n", + "_________________________________________________________________\n", + "Layer (type) Output Shape Param # \n", + "=================================================================\n", + "conv2d_1 (Conv2D) (None, 98, 98, 32) 896 \n", + "_________________________________________________________________\n", + "max_pooling2d_1 (MaxPooling2 (None, 49, 49, 32) 0 \n", + "_________________________________________________________________\n", + "conv2d_2 (Conv2D) (None, 47, 47, 64) 18496 \n", + "_________________________________________________________________\n", + "max_pooling2d_2 (MaxPooling2 (None, 23, 23, 64) 0 \n", + "_________________________________________________________________\n", + "conv2d_3 (Conv2D) (None, 21, 21, 128) 73856 \n", + "_________________________________________________________________\n", + "max_pooling2d_3 (MaxPooling2 (None, 10, 10, 128) 0 \n", + "_________________________________________________________________\n", + "conv2d_4 (Conv2D) (None, 8, 8, 128) 147584 \n", + "_________________________________________________________________\n", + "max_pooling2d_4 (MaxPooling2 (None, 4, 4, 128) 0 \n", + "_________________________________________________________________\n", + "flatten_1 (Flatten) (None, 2048) 0 \n", + "_________________________________________________________________\n", + "dense_1 (Dense) (None, 512) 1049088 \n", + "_________________________________________________________________\n", + "dense_2 (Dense) (None, 3) 1539 \n", + "=================================================================\n", + "Total params: 1,291,459\n", + "Trainable params: 1,291,459\n", + "Non-trainable params: 0\n", + "_________________________________________________________________\n" + ] + } + ], + "source": [ + "#建立cnn\n", + "from keras import layers\n", + "from keras import models\n", + "\n", + "model = models.Sequential()\n", + "model.add(layers.Conv2D(32, (3, 3), activation='relu',\n", + " input_shape=(100, 100, 3)))\n", + "model.add(layers.MaxPooling2D((2, 2)))\n", + "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n", + "model.add(layers.MaxPooling2D((2, 2)))\n", + "model.add(layers.Conv2D(128, (3, 3), activation='relu'))\n", + "model.add(layers.MaxPooling2D((2, 2)))\n", + "model.add(layers.Conv2D(128, (3, 3), activation='relu'))\n", + "model.add(layers.MaxPooling2D((2, 2)))\n", + "model.add(layers.Flatten())\n", + "model.add(layers.Dense(512, activation='relu'))\n", + "model.add(layers.Dense(3, activation='softmax'))\n", + "\n", + "model.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "#優化器\n", + "from keras import optimizers\n", + "\n", + "model.compile(loss='categorical_crossentropy',\n", + " optimizer=optimizers.RMSprop(lr=1e-4),\n", + " metrics=['acc'])" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Found 60 images belonging to 3 classes.\n", + "Found 27 images belonging to 3 classes.\n" + ] + } + ], + "source": [ + "#ImageDataGenerator套件label圖片\n", + "from keras.preprocessing.image import ImageDataGenerator\n", + "import numpy as np\n", + "train_datagen = ImageDataGenerator(rescale=1./255)\n", + "test_datagen = ImageDataGenerator(rescale=1./255)\n", + "\n", + "train_generator = train_datagen.flow_from_directory(\n", + " train_dir,\n", + " target_size=(100, 100),\n", + " batch_size=20,\n", + " class_mode='categorical')\n", + "\n", + "validation_generator = test_datagen.flow_from_directory(\n", + " validation_dir,\n", + " target_size=(100, 100),\n", + " batch_size=20,\n", + " class_mode='categorical')" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "data batch shape: (20, 100, 100, 3)\n", + "labels batch shape: (20, 3)\n" + ] + } + ], + "source": [ + "for data_batch, labels_batch in train_generator:\n", + " print('data batch shape:', data_batch.shape)\n", + " print('labels batch shape:', labels_batch.shape)\n", + " break" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 1/30\n", + "100/100 [==============================] - 94s 940ms/step - loss: 0.6316 - acc: 0.7760 - val_loss: 0.8940 - val_acc: 0.7778\n", + "Epoch 2/30\n", + "100/100 [==============================] - 93s 930ms/step - loss: 0.4135 - acc: 0.8530 - val_loss: 0.6577 - val_acc: 0.7037\n", + "Epoch 3/30\n", + "100/100 [==============================] - 94s 941ms/step - loss: 0.2813 - acc: 0.8860 - val_loss: 0.6228 - val_acc: 0.6296\n", + "Epoch 4/30\n", + "100/100 [==============================] - 93s 933ms/step - loss: 0.1676 - acc: 0.9280 - val_loss: 1.7445 - val_acc: 0.5926\n", + "Epoch 5/30\n", + "100/100 [==============================] - 94s 939ms/step - loss: 0.1259 - acc: 0.9370 - val_loss: 1.0945 - val_acc: 0.5926\n", + "Epoch 6/30\n", + "100/100 [==============================] - 94s 942ms/step - loss: 0.1062 - acc: 0.9405 - val_loss: 2.6368 - val_acc: 0.5926\n", + "Epoch 7/30\n", + "100/100 [==============================] - 96s 960ms/step - loss: 0.0993 - acc: 0.9390 - val_loss: 1.2031 - val_acc: 0.5926\n", + "Epoch 8/30\n", + "100/100 [==============================] - 95s 954ms/step - loss: 0.0951 - acc: 0.9305 - val_loss: 2.8705 - val_acc: 0.5926\n", + "Epoch 9/30\n", + "100/100 [==============================] - 96s 957ms/step - loss: 0.0892 - acc: 0.9385 - val_loss: 2.4154 - val_acc: 0.5926\n", + "Epoch 10/30\n", + "100/100 [==============================] - 97s 966ms/step - loss: 0.0860 - acc: 0.9365 - val_loss: 4.6667 - val_acc: 0.5926\n", + "Epoch 11/30\n", + "100/100 [==============================] - 97s 970ms/step - loss: 0.0860 - acc: 0.9365 - val_loss: 2.0964 - val_acc: 0.5926\n", + "Epoch 12/30\n", + "100/100 [==============================] - 97s 973ms/step - loss: 0.0844 - acc: 0.9365 - val_loss: 4.6467 - val_acc: 0.5926\n", + "Epoch 13/30\n", + "100/100 [==============================] - 97s 969ms/step - loss: 0.0842 - acc: 0.9365 - val_loss: 0.8492 - val_acc: 0.5926\n", + "Epoch 14/30\n", + "100/100 [==============================] - 97s 972ms/step - loss: 0.0811 - acc: 0.9345 - val_loss: 6.7441 - val_acc: 0.5926\n", + "Epoch 15/30\n", + "100/100 [==============================] - 97s 971ms/step - loss: 0.0801 - acc: 0.9380 - val_loss: 1.3565 - val_acc: 0.5926\n", + "Epoch 16/30\n", + "100/100 [==============================] - 97s 973ms/step - loss: 0.0775 - acc: 0.9315 - val_loss: 5.6226 - val_acc: 0.5926\n", + "Epoch 17/30\n", + "100/100 [==============================] - 98s 983ms/step - loss: 0.0816 - acc: 0.9315 - val_loss: 5.0703 - val_acc: 0.5926\n", + "Epoch 18/30\n", + "100/100 [==============================] - 98s 984ms/step - loss: 0.0746 - acc: 0.9330 - val_loss: 7.8723 - val_acc: 0.5926\n", + "Epoch 19/30\n", + "100/100 [==============================] - 98s 983ms/step - loss: 0.0747 - acc: 0.9360 - val_loss: 2.2070 - val_acc: 0.5926\n", + "Epoch 20/30\n", + "100/100 [==============================] - 98s 979ms/step - loss: 0.0756 - acc: 0.9320 - val_loss: 8.8935 - val_acc: 0.5926\n", + "Epoch 21/30\n", + "100/100 [==============================] - 98s 983ms/step - loss: 0.0775 - acc: 0.9350 - val_loss: 2.1388 - val_acc: 0.5926\n", + "Epoch 22/30\n", + "100/100 [==============================] - 98s 984ms/step - loss: 0.0734 - acc: 0.9360 - val_loss: 4.9511 - val_acc: 0.5926\n", + "Epoch 23/30\n", + "100/100 [==============================] - 98s 982ms/step - loss: 0.0729 - acc: 0.9335 - val_loss: 1.6979 - val_acc: 0.5926\n", + "Epoch 24/30\n", + "100/100 [==============================] - 99s 988ms/step - loss: 0.0718 - acc: 0.9325 - val_loss: 3.6615 - val_acc: 0.5926\n", + "Epoch 25/30\n", + "100/100 [==============================] - 98s 983ms/step - loss: 0.0729 - acc: 0.9375 - val_loss: 6.6448 - val_acc: 0.5926\n", + "Epoch 26/30\n", + "100/100 [==============================] - 98s 984ms/step - loss: 0.0721 - acc: 0.9355 - val_loss: 6.3982 - val_acc: 0.5926\n", + "Epoch 27/30\n", + "100/100 [==============================] - 98s 980ms/step - loss: 0.0711 - acc: 0.9340 - val_loss: 12.2808 - val_acc: 0.5926\n", + "Epoch 28/30\n", + "100/100 [==============================] - 98s 980ms/step - loss: 0.0721 - acc: 0.9365 - val_loss: 7.8047 - val_acc: 0.5926\n", + "Epoch 29/30\n", + "100/100 [==============================] - 98s 980ms/step - loss: 0.0704 - acc: 0.9310 - val_loss: 9.7293 - val_acc: 0.5926\n", + "Epoch 30/30\n", + "100/100 [==============================] - 98s 982ms/step - loss: 0.0737 - acc: 0.9360 - val_loss: 6.4731 - val_acc: 0.5926\n" + ] + } + ], + "source": [ + "#epoch訓練30\n", + "history = model.fit_generator(\n", + " train_generator,\n", + " steps_per_epoch=100,\n", + " epochs=30,\n", + " validation_data=validation_generator,\n", + " validation_steps=50)" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "model.save('train1.h5')" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtcAAAEjCAYAAAD5ZS3PAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3zV5fXA8c/JIpOEDJCw90amoCDDCYqCaFUUK1bKT+usxaqtdbaO1oFWa4sDF4ooDlSstoqiULaIbJAZwkgCJCGDrPP747mBEDNuyL1ZnPfrdV/33u94vucGTc59vud5HlFVjDHGGGOMMdUXUNsBGGOMMcYY01BYcm2MMcYYY4yPWHJtjDHGGGOMj1hybYwxxhhjjI9Ycm2MMcYYY4yPWHJtjDHGGGOMj1hybYwxpYjIZyJybW3HURvEmSEiB0VkaW3HY4wx9Y0l18aYektEtotIjogc9iSDn4pIq+q2q6qjVfW1E4hHRORWEVkjIlkikiQi74pIL8/+V0VEReS0Eud0FBEt8f5rEckt+TlE5BwR2V7BddVzvcMisltEnhKRwKrG7zEUOBdoqaqnVXawMcaY41lybYyp7y5S1UigObAP+HstxvIMcBtwKxALdAY+BC4sccwB4M+VtJMF/KmK1z7V83M4G7gK+HUVz0dEgoA2wHZVzTrB840x5qRmybUxpkFQ1VzgPaB78TYRuVBEvheRDBHZJSIPlNgXKiJvikiaiBwSkWUi0syz72sRmVzi2F+LyHoRyRSRdSLSr/T1RaQTcBMwQVW/UtUjqpqtqjNV9bESh74G9BaR4RV8nGeBCSLS8QR+DhuAb4GenrgSRWSOiKSIyDYRubVEzA+IyHuen0MGcD3wEnC6pxf8wRKff4uIHBCRuSKSWKINFZGbRGQzsLnEtt+IyGbPz+xhEekgIv/z/FvMFpEQz7FNROQTT3wHPa9blmj/a8/5Cz1tfSEi8SX2DxWRRZ5/w10iMsmzvZGIPCEiO0Vkn4j8U0TCqvrzNMaYqrLk2hjTIIhIOHAFsLjE5izgl0AMrvf4RhEZ59l3LRANtALigBuAnDLa/QXwgKedxsDFQFoZIZwNJKlqZXXK2cAjwF8qOGY38KLnulUiIt2BM4HvRSQA+Bj4AWjhifF2ETm/xCljcV9KYoDXcT+H/6lqpKreLyJnAY8Cl+PuDuwAZpW67DhgECW+2ACjgP7AYOD3wHTgatzPuycwwXNcADAD12PeGvdv8Fyp9q8CrgOaAiHAVM9nbQ18hrtbkQD0AVZ5znkcd+egD9DR8/nvq+hnZ4wxvmDJtTGmvvtQRA4BGbha4b8V71DVr1X1R1UtUtXVwNtAcY9xPi6p7qiqhaq6QlUzymh/MvBXVV2mzhZV3VHGcXHAHi9j/hfQWkRGV3DMo8BFItLDyzZXishBXDL9Ei5hHQgkqOpDqpqnqltxSfuVJc77n6p+6PkZ/ezLBS4hfkVVV6rqEeAeXM9225KxquqBUuc/rqoZqroWWAN8oapbVTUdlxD3BVDVNFWd4+nlz8R96Sjdqz9DVTd52p+NS5iLY/uvqr6tqvmetlaJiODKYn7riSsT94XmSowxxs+sPs4YU9+NU9X/egbwjQW+EZHuqrpXRAYBj+F6SkOARsC7nvPewPWizhKRGOBN4I+qml+q/VbAT17EkYbr2a2Uqh4RkYeBhznWg1v6mBQReQ54CHjBi2b7qeqWkhtEpA2Q6PnyUSwQVzZSbFcl7SYCK0vEdVhE0nA9wdsraGNfidc5Zbw/xRNjOPA0rqe7iWd/lIgEqmqh5/3eEudmA5Ge1+X92yQA4cAKl2cDILjPbowxfmU918aYBsHT+/w+UIib8QLgLWAu0EpVo4F/4pIsPD2dD6pqd+AMYAyu9KO0XUAHL0L4EmgpIgO8DHkGrizlkgqO+RswEldecSJ2AdtUNabEI0pVLyhxjJZ3skcyrmQDABGJwPXS765CGxX5HdAFGKSqjYFhxZfy4tzy/m1ScQl8jxKfO9oz4NMYY/zKkmtjTIMgzlhc7+d6z+Yo4ICq5nqmv7uqxPEjRaSXp8c7A1cmUli6XVyJxVQR6e+5RkdPj/BxVHUz8A/gbREZISIhnkGTV4rI3WUcX4Crqb6rvM+kqoeAJ3E1yydiKZAhIneJSJiIBIpITxEZWIU23gKuE5E+ItIIV16xRFW3n2BMpUXhEuFDIhIL3F+Fc2cC54jI5SISJCJxItJHVYtw5S9Pi0hTABFpUarW3Bhj/MKSa2NMffexiBzGJch/Aa711PkC/AZ4SEQycYPZZpc47xTcQL4MXDL+Da405Diq+q6n3beATNzUerHlxHIrbjDe88AhXMnCJbg66LK8TeV12s9QdtJfKU9ZxUW4GuVtuB7dl3A95t628SVuWsA5nlg74Nva5WlAmCe2xcC/qxDbTuACXO/3AdxgxlM9u+8CtgCLPTOh/BfXQ26MMX4lqtW5m2eMMcYYY4wpZj3XxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXxhhjjDHG+Igl18YYY4wxxviIJdfGGGOMMcb4iCXXpl4Rka9F5KCINKrtWIwxxtQ+EdkuIufUdhzGFLPk2tQbItIWOBNQ4OIavG5QTV3LGGOMMfWbJdemPvklsBh4Fbi2eKOIhInIkyKyQ0TSReQ7EQnz7BsqIotE5JCI7BKRSZ7tX4vI5BJtTBKR70q8VxG5SUQ2A5s9257xtJEhIitE5MwSxweKyB9E5CcRyfTsbyUiz4vIkyU/hIh8LCK3++MHZIwxxhGRX4vIFhE5ICJzRSTRs11E5GkR2e/5m7FaRHp69l0gIus8v8d3i8jU2v0Upj6y5NrUJ78EZnoe54tIM8/2J4D+wBlALPB7oEhEWgOfAX8HEoA+wKoqXG8cMAjo7nm/zNNGLPAW8K6IhHr23QFMAC4AGgO/ArKB14AJIhIAICLxwNnA21X54MYYY7wnImcBjwKXA82BHcAsz+7zgGFAZyAGuAJI8+x7Gfg/VY0CegJf1WDYpoGw5NrUCyIyFGgDzFbVFcBPwFWepPVXwG2qultVC1V1kaoeAa4G/quqb6tqvqqmqWpVkutHVfWAquYAqOqbnjYKVPVJoBHQxXPsZOBeVd2ozg+eY5cC6biEGuBK4GtV3VfNH4kxxpjyXQ28oqorPX8P7gFO95QX5gNRQFdAVHW9qu7xnJcPdBeRxqp6UFVX1kLspp6z5NrUF9cCX6hqquf9W55t8UAoLtkurVU52721q+QbEfmdiKz33EY8BER7rl/ZtV4DJnpeTwTeqEZMxhhjKpeI660GQFUP43qnW6jqV8BzwPPAPhGZLiKNPYdeirsDuUNEvhGR02s4btMAWHJt6jxP/fTlwHAR2Ssie4HfAqfibvflAh3KOHVXOdsBsoDwEu9PKeMYLRHDmcBdnjiaqGoMrkdavLjWm8BYETkV6AZ8WM5xxhhjfCMZd7cTABGJAOKA3QCq+qyq9gd64MpD7vRsX6aqY4GmuN/Vs2s4btMAWHJt6oNxQCGu9rmP59EN+BZXh/0K8JSIJHoGFp7umapvJnCOiFwuIkEiEicifTxtrgLGi0i4iHQErq8khiigAEgBgkTkPlxtdbGXgIdFpJNnsExvEYkDUNUkXL32G8Cc4jITY4wxPhMsIqHFD1xSfJ2I9PH8PXgEWKKq20VkoIgMEpFgXEdLLlAoIiEicrWIRKtqPpCB+9tjTJVYcm3qg2uBGaq6U1X3Fj9wt/WuBu4GfsQlsAeAx4EAVd2Ju733O8/2VbjeboCngTxgH65sY2YlMXyOGxy5CXerMZfjy0aewv0y/wL3C/llIKzE/teAXlhJiDHG+MM8IKfE40zgT8AcYA/uzuKVnmMbAy8CB3G/z9NwA+MBrgG2i0gGcAPHSvqM8ZqoauVHGWOqRUSG4cpD2qpqUW3HY4wxxhj/sJ5rY/zMc+vxNuAlS6yNMcaYhs2Sa2P8SES6AYdwAy+n1XI4xhhjjPEzKwsxxhhjjDHGR6zn2hhjjDHGGB8Jqu0AfCU+Pl7btm1b22EYYwwrVqxIVdWE2o6jIbLf9caYuqCi3/MNJrlu27Yty5cvr+0wjDEGEdlR+VHmRNjvemNMXVDR73m/loWIyCgR2SgiW0Tk7jL2txGRL0VktYh8LSItS+y7VkQ2ex7X+jNOY4wxxhhjfMFvybWIBALPA6NxK+tNEJHupQ57AnhdVXsDDwGPes6NBe4HBgGnAfeLSBN/xWqMMcYYY4wv+LPn+jRgi6puVdU8YBYwttQx3YEvPa/nl9h/PvAfVT2gqgeB/wCj/BirMcYYY4wx1ebPmusWHL88dBKuJ7qkH4BLgWeAS4AoEYkr59wW/gvVGGNMfZWfn09SUhK5ubm1HYqpgtDQUFq2bElwcHBth2KMT/kzuZYytpWeVHsq8JyITAIWALuBAi/PRUSmAFMAWrduXZ1YjTHG1FNJSUlERUXRtm1bRMr682HqGlUlLS2NpKQk2rVrV9vhGONT/iwLSQJalXjfEkgueYCqJqvqeFXtC/zRsy3dm3M9x05X1QGqOiAhwWa9MsaYk1Fubi5xcXGWWNcjIkJcXJzdbTANkj+T62VAJxFpJyIhwJXA3JIHiEi8iBTHcA/wiuf158B5ItLEM5DxPM82Y4wpl6qyPTWL7alZPmlrmw/aMTXDEuv6x/7NTJ3w03zIOejTJv1WFqKqBSJyMy4pDgReUdW1IvIQsFxV5wIjgEdFRHFlITd5zj0gIg/jEnSAh1T1gL9iNcbUT0VFyqb9mSzdduDoY3/mEQBObRnNpf1bclHvRJpEhHjd5vbULOb+kMxHq3azLTWLxfecTdPGof76CMYYY2rLntXw9pXQ4xK45J8+a9avi8io6jxgXqlt95V4/R7wXjnnvsKxnmxjjCG/sIi1yRks23aAJdsOsGz7AdJz8gFoHh3K6R3iOK1dLDl5hcxZuZv7PlrLw5+s46yuTbm0X0tGdm1KcODPb9jtz8zl09V7+HBVMj/sOgTAoHaxXD+0PeGNGsxaWzVKRF4BxgD7VbVnBccNBBYDV3j+JtQ7aWlpnH322QDs3buXwMBAiksVly5dSkhI5V/urrvuOu6++266dOni1TVfeukl1qxZw7Rp0048cGNOZjkHYfY1ENYEzn3Ip03bXw1jTJ2TX1jEjrQsNu87zKZ9h9m0P5Mt+w6zNfUw+YVubHO7+AhG9TiF09rFclq7WFo2CTvuNvPkM9uzLjmDOSuT+GjVbj5fu4/YiBAuPjWRy/q3pHVcOJ+v2cvcH5JZuCWVIoXuzRvzhwu6MqZ3IokxYbX18RuKV4HngNfLO8CzHsLj1POyv7i4OFatWgXAAw88QGRkJFOnTj3uGFVFVQkIKLsac8aMGX6P0xjjUVQE70+B9N1w3TyIbOrT5i25NsZUS0FhEYX6s8l8jqMKOXmFHD5SQEZuPodzCzh8xD0yc4sf+ew4kP2zJFoEWjUJp3OzSM7q1pQeiY05rV0sTaMqL9XontiY7onduXt0VxZsSmHOyiTeWrKTVxdtJzBAKCxSWseGc9PIjlx8aiKdmkX55GdiQFUXiEjbSg67BZgDDPR7QLVgy5YtjBs3jqFDh7JkyRI++eQTHnzwQVauXElOTg5XXHEF993nbuYOHTqU5557jp49exIfH88NN9zAZ599Rnh4OB999BFNm3r3x//NN9/k8ccfR1W5+OKLeeSRRygoKOC6665j1apVqCpTpkzh1ltv5emnn+bFF18kODiYXr168eabb/rzx2FM3bHgb7D5C7jgCWh1ms+bt+TaGFNl21Kz+HL9Pr7asJ+l2w5QUFRxcu2NwAChRUwYnZtFMrJrUzo3i6Rzsyg6JEQSFhJYrbaDAwM4u1szzu7WjPTsfD5enUzSwRzO79GMPq1ibGBVLRCRFrj1Dc6ikuS6KtOuPvjxWtYlZ/goSqd7YmPuv6jHCZ27bt06ZsyYwT//6eo5H3vsMWJjYykoKGDkyJFcdtlldO9+/OLF6enpDB8+nMcee4w77riDV155hbvvvrvSayUlJXHvvfeyfPlyoqOjOeecc/jkk09ISEggNTWVH3/8EYBDh1zp01//+ld27NhBSEjI0W3GNHib/wNfPwq9r4SBk/1yCUuujWlADmblMWdlEltTsxjcPo4zO8ZXaTBfefILi1i2/QBfrd/PVxv2s9Uzi0aXZlFcN6QtMeGVXyM8JJDIRkFEhQYTFRpEZKMgIkODiAoNIqpRMKHBATWS5EaHBzNxcBu/X8dUahpwl6oWVvbvrqrTgekAAwYMqP43uRrUoUMHBg489t3h7bff5uWXX6agoIDk5GTWrVv3s+Q6LCyM0aNHA9C/f3++/fZbr661ZMkSzjrrLOLj4wG46qqrWLBgAXfddRcbN27ktttu44ILLuC8884DoEePHkycOJGxY8cybtw4X3xcY+q2g9thzmRo1gPGPO1ujfqBJdfG1HOqysqdh5i5eAef/LiHvIIiIkICeWvJTkSgd4tohnVOYFjnBPq2iiGojAF9pds7lJ3ProPZbNp3mPkb97NgYwqZRwoICQzg9A5xTBrSlpFdmtIqNryGPqVpgAYAszyJdTxwgYgUqOqH1Wn0RHuY/SUiIuLo682bN/PMM8+wdOlSYmJimDhxYpnzPJccABkYGEhBQYFX19JyyrPi4uJYvXo1n332Gc8++yxz5sxh+vTpfP7553zzzTd89NFH/PnPf2bNmjUEBlbvLpExdVZ+DrxzDaBwxRsQ4r+/X5ZcG1NPHT5SwIff72bmkp2s35NBZKMgrhjQiqsGtaZzsyh+3J3ONxtTWLA5hefnb+HvX20hKjSIIR3iGdY5gd4to9mXkcuuA9nsOpjDrgPZ7DyQTdLBHA4fOfbHPCGqERf2bs5ZXZsypGM8ETZ7hvEBVT26LJ+IvAp8Ut3Euq7LyMggKiqKxo0bs2fPHj7//HNGjRrls/YHDx7MnXfeSVpaGtHR0cyaNYupU6eSkpJCaGgov/jFL2jXrh033HADhYWFJCUlcdZZZzF06FBmzpxJdnY2UVE27sA0QKrw6e9g72q4ajbEtvfr5eyvpDH1zLrkDGYu2cGH3+8mK6+Q7s0b88glvbi4TyKRJRLfPq1i6NMqhtvO6UR6dj4Lf0plwaYUFmxK4d9r9x7XZmhwAK2ahNMqNpzB7eNo2SSMVrHhtI2LoFPTSAICrCbZVI2IvI1byyBeRJKA+4FgAFX13YSy9Ui/fv3o3r07PXv2pH379gwZMqRa7b388su8996x2QuXL1/OQw89xIgRI1BVLrroIi688EJWrlzJ9ddfj6oiIjz++OMUFBRw1VVXkZmZSVFREXfddZcl1qZu2L8BvvgjjP4rxHXwTZsrZsCqmTD8Luh8vm/arICUdxupvhkwYIAuX768tsMwxueOFBSybNtBFmx2ifGGvZk0CgpgTO9EJg5uXeUBearKlv1uirvmMaG0ahJOfGSIDerzIRFZoaoDajuOhqis3/Xr16+nW7dutRSRqQ77tzM/8841sH4uNOsJ1/+n+uUbScvhlVHQfrjrtQ7wTelTRb/nrefamDpGVdmamnW0l3nx1gPk5BcSHCgMaBPLfWO6M75fC68GEZZFROjULMqmnTPGGFO3pG6G9R9Du+GwbQHMmwpjnz/xgYdZqTD7l9C4OYx/0WeJdWUsuTamGgoKi1i/J5O9GbkczM7jUHYeB7PzOZSdx4GsY68PZecTGCAlZskIJqpRkGf2DDdrRmSjILamZvHNxhR2H8oBoG1cOJcPaMmwzgkMbh9n9c7GGGMarkXPQlAjuPQlWPYSfPM4tBoE/a+telsFR+C96yA7Da7/AsJjfR9vOewvtTFVUFSkbNyXyaKf0li0JZUl2w4cN/gPIDhQiAkPoUl4MDHhIbSLjyAmLIQi1aOLpqTn5LP7YDaZnsVUsvMKAYgICeSMjvHcMKIDwzsl0DrOZuMwxhhzEsjYAz/Mgr7XuBUTh98FSctg3p3QvDck9vW+rSOH4Z2Jrvd73D+h+an+i7sMllwbUwFVZUdaNgt/SmXRT2ks/imNtKw8wC2/fXGfRE5vH0fbuAhiwoNpEhFCREhgleuXCwqLyDpSSHijQIIrmSrPGGOMaXAW/wOKCuCMW9z7gEAY/xL8a5gr7ZjyjXe9zzkHYeYvYPcKGPcC9Jng37jLYMm1MaWk5+SzcIubWePbzalHSzSaNW7E8M4JnNExntM7xNEiJsxn1wwKDCA63JJqY4wxJ6GcQ7B8BvS4BGLbHdseEQeXv+YGJH5wA0yYBQEV/K3M3AdvjofUTXD569DtIv/HXgZLrs1Jr7BIj5sTetWuQxQWKVGNgjijYxw3jOjAGR3iaB8fYTNqGGOMMb62/GXIy4Qht/98X8sBMOpRN7jxuydh2J1lt3FoJ7w+FjL3ullBOoz0b8wVsK4yc9LJzM3nx6R0Zi/bxc1vraT/n//DuOcXMu3LTRQUFvGbER1494bTWXnfufzrmgFcM7gNHRIiLbE2xpRpxIgRfP7558dtmzZtGr/5zW8qPC8yMhKA5ORkLrvssnLbrmya2WnTppGdnX30/QUXXMChQ4e8Cb1CDzzwAE888US12zGmQvk5sPif0OFsV1tdloGTodcv4Ku/wE/zf74/ZZPr3c5Og19+VKuJNVjPtWmgjhQUsjMtm62pWWxLzWJbiuc5LYuUzCNHj0uIasTZXZsxrHM8Z3ZKIDbixKa3M8acvCZMmMCsWbM4//xji1PMmjWLv/3tb16dn5iYeNxiMFU1bdo0Jk6cSHi4GwA9b968E27LmBq36i3I2g9Df1v+MSJw0TOwdw3MuR7+71uIbuH27fkB3hjvjpn0KZzSq2biroAl16beKSgsYn/mEfak57AnPZc9h3Ldc/H79Bz2Zx6h5PpI8ZFu1o6RXRJoFx9Ju/gIOjaNsB5pY0y1XXbZZdx7770cOXKERo0asX37dpKTkxk6dCiHDx9m7NixHDx4kPz8fP785z8zduzY487fvn07Y8aMYc2aNeTk5HDdddexbt06unXrRk5OztHjbrzxRpYtW0ZOTg6XXXYZDz74IM8++yzJycmMHDmS+Ph45s+fT9u2bVm+fDnx8fE89dRTvPLKKwBMnjyZ22+/ne3btzN69GiGDh3KokWLaNGiBR999BFhYd6NIymrzaysLC6//HKSkpIoLCzkT3/6E1dccQV33303c+fOJSgoiPPOO896ws3xCgvc9Hst+kPboRUfGxIBV7wB00fAu9fCpHlu0OJbl0NotOux9tWKjtXk1+RaREYBzwCBwEuq+lip/a2B14AYzzF3q+o8EWkLrAc2eg5drKo3+DNWUzfl5BWybk86q5OKH4fYlppFUamFRcNDAmkeHUpiTBidmibQPCaM9vERtIuPoG18BNFhwbXzAYwxNeuzu2Hvj75t85ReMPqxcnfHxcVx2mmn8e9//5uxY8cya9YsrrjiCkSE0NBQPvjgAxo3bkxqaiqDBw/m4osvLvdL/QsvvEB4eDirV69m9erV9OvX7+i+v/zlL8TGxlJYWMjZZ5/N6tWrufXWW3nqqaeYP38+8fHxx7W1YsUKZsyYwZIlS1BVBg0axPDhw2nSpAmbN2/m7bff5sUXX+Tyyy9nzpw5TJw4sdIfRXltbt26lcTERD799FMA0tPTOXDgAB988AEbNmxARHxSqmI8jhx2M2IMvB56lV1SVC+s/wgObodzH/ZuoZj4Tm5RmXevhXeuhm3fQnRL+OWH7rmO8FtyLSKBwPPAuUASsExE5qrquhKH3QvMVtUXRKQ7MA9o69n3k6r28Vd8puaoKpv2HSb18BECA4TgQCEoIICgEs/BnueUzCOs3p3Oj0mHWJ2Uzub9hyn0ZNLNGjeiV4sYLujVnMSYME6JDiUx2j03Dg2yHmhjTK0pLg0pTq6Le3ZVlT/84Q8sWLCAgIAAdu/ezb59+zjllFPKbGfBggXceuutAPTu3ZvevY/VoM6ePZvp06dTUFDAnj17WLdu3XH7S/vuu++45JJLiIiIAGD8+PF8++23XHzxxbRr144+fdyf2P79+7N9+3avPmd5bY4aNYqpU6dy1113MWbMGM4880wKCgoIDQ1l8uTJXHjhhYwZM8araxgvrJ8LOxdB8vfQtDs0617bEVWdKnw3DeI6Qdcq/LfRYxzsugkWP++++E78ACIT/BfnCfBnz/VpwBZV3QogIrOAsUDJ5FqBxp7X0UCyH+MxNehgVh7feqazW7Aphf0l6py9ERsRQu+W0ZzXvRm9WsbQu2U0zRqH+ilaY0yDUUEPsz+NGzeOO+64g5UrV5KTk3O0x3nmzJmkpKSwYsUKgoODadu2Lbm5uRW2VVZHwbZt23jiiSdYtmwZTZo0YdKkSZW2o6rl7mvUqNHR14GBgceVn5xIm507d2bFihXMmzePe+65h/POO4/77ruPpUuX8uWXXzJr1iyee+45vvrqK6+uYyrxwyyIbgWFea4X99fzoVFkbUdVNT99BXtXw8V/r3h6vbKc+yAk9oHO57uSkDrGn8l1C2BXifdJwKBSxzwAfCEitwARwDkl9rUTke+BDOBeVf229AVEZAowBaB169a+i9xUWUFhEd/vOnQ0mV69Ox1ViA4LZmineIZ3SqBtfAQFhUXkFymFRUXkFyoFhUpBUdHR58ahwfRqGU2LmDDriTbG1BuRkZGMGDGCX/3qV0yYcGzRivT0dJo2bUpwcDDz589nx44dFbYzbNgwZs6cyciRI1mzZg2rV68GICMjg4iICKKjo9m3bx+fffYZI0aMACAqKorMzMyflYUMGzaMSZMmcffdd6OqfPDBB7zxxhvV+pzltZmcnExsbCwTJ04kMjKSV199lcOHD5Odnc0FF1zA4MGD6dixY7WubTzSd7uVB0fcDW3OcNPPffJbGD/du9KKumLhNIhqDr2vqPq5gcHQ+3Lfx+Qj/kyuy/oXLv2VdwLwqqo+KSKnA2+ISE9gD9BaVdNEpD/woYj0UNWM4xpTnQ5MBxgwYED5XyZaUjEAACAASURBVNGNX+TmF/L1xhQ+/iGZBZtTyMwtIECgT6sYbj+7M8M6x9O7ZQyBAfXof3ZjjDlBEyZMYPz48cyaNevotquvvpqLLrqIAQMG0KdPH7p27VphGzfeeCPXXXcdvXv3pk+fPpx22mkAnHrqqfTt25cePXrQvn17hgwZcvScKVOmMHr0aJo3b878+cemKevXrx+TJk062sbkyZPp27ev1yUgAH/+85+ZNm3a0fdJSUlltvn5559z5513EhAQQHBwMC+88AKZmZmMHTuW3NxcVJWnn37a6+uaCvw4G1CXXMa2hxH3wPy/QNsh0H9SbUfnnd0r3BeEcx+GoEaVH1/PSEW3jarVsEuWH1DV8z3v7wFQ1UdLHLMWGKWquzzvtwKDVXV/qba+BqaqarmTfQ4YMEArmwvUVF9hkbJkaxofrUpm3po9ZOYWEBcRwjndmjG8SwJDOsQTHW6DB83JTURWqOqA2o6jISrrd/369evp1q1bLUVkqsP+7apIFf4x2JVCXP+F21ZUCG9eCjsWwa+/rBNT0VXqnWtg2zdw+xoIbVz58XVQRb/n/dlzvQzoJCLtgN3AlcBVpY7ZCZwNvCoi3YBQIEVEEoADqlooIu2BTsBWP8ZqKqCqrNmdwYerdvPxD8nszzxCREgg5/c8hbF9WjCkQxxBgbYekTHGGONXe36AlA0wpsRdgIBAGP8i/OtMmH0tTPm6biesqVtg/cdw5h11O85q8FtyraoFInIz8Dlumr1XVHWtiDwELFfVucDvgBdF5Le4kpFJqqoiMgx4SEQKgELgBlU94K9YG5K96bms3HmQDXsyOHykkJz8Qo7ku+fco89F5OYXkldQRHBgAKEhgYQGBRAWEkhoUKB7Dg4gNDgQQfh64362pmYRHCiM6NKUcX1acFbXpoSFBNb2xzXGGGNOHqvfgcAQ6HHJ8dsjE+DSl+G1MfDxbXDZK3W3/nrRM64UZFDDnWHZr/Ncq+o83PR6JbfdV+L1OmBIGefNAeb4M7aGIK+giLXJ6azceYiVOw/y/Y6DJKe70eMBAuEhQYQGu0Q5LDjwaPLcOCyYplGNCAkKIL+wiNz8InLyCzmYlXc0+c7JLyQ3r5AjhUX0b92EKcPaM7pncyv5MMbUSapqg6DrGX+VpTZYhQXw47tuhoywJj/f33YInHUvfPmQez1wcs3HWFJhAeQd9jyy3HNWmpvppO81ENm0duPzI1uhsY5SVTKPFHAoK5+D2XkczM7jULZ7nXwoh5U7D/Hj7nTyCooAaBETRr82TZjcugn92jShe/PGhARZqYYxpuELDQ0lLS2NuLg4S7DrCVUlLS2N0FCbYtVrP30FWSlw6oTyjxnyW9jxP/j3PdBigJuuriaowndPwYrXXBJ95DAUljMFb0AwnHFLzcRVSyy5rgNUleU7DjJnRRIrdhw8mkgXlF6G0CMkKIBeLaK59vQ29PMk0zYHtDHmZNWyZUuSkpJISUmp7VBMFYSGhtKyZd1ZVa/OWz0LwmKh47nlHxMQAJf8y9Vfv3st/N8C/88DXVQIn94BK16FdsMhrqNbqrxRlHsOiTz23CgSGreA2Hb+jamWWXJdi3YdyOb9lbt5//skdqRlEx4SyBkd4hnQNpYm4cE0CQ8hxvPcJCKYmPAQmoSHEB0WbNPbGWOMR3BwMO3aNew/1uYkl5sBGz6FvhMhKKTiYyPiXM31jAvgo5vh8tf9V3+dnwNzJsOGT2DoHXD2fXW31rsGWXJdww4fKWDe6j3MWZnEkm1ujObp7eO45axOjO55ChGN7J/EGGOMMSWs+wgKcisuCSmp9WA45374z32w8jX/zH+dfQDengC7lsDov8Kg//P9Neopy+RqgKqyZNsBZi3dyb/X7iU3v4h28RFMPa8z4/q2oGWT8NoO0RhjjDF11ep3ILYDtOjv/Tmn3wIbP4OvH3dJuS8Xa0lPcnNrH9jqesl7jvdd2w2AJdd+lFdQxLwf9/DSd1tZszuDqNAgxvdryaX9WtKvdYwNvDHGNFgi8gowBtivqj3L2H81cJfn7WHgRlX9oQZDNKZ+OLQTtn8LI/9YtZKLgAAYfhe8MQ5WzYQBv/JNPPs3wJvj4UgmTJwD7Yb5pt0GxJJrP0jPzuetpTt5bdF29mbk0iEhgkcu6cX4fi0IDba5oY0xJ4VXgeeA18vZvw0YrqoHRWQ0MB0YVEOxGVN/rJ7tnntfXvVz249ws4Z897Sb/i6wmtPp7lwMb13hesGvm1c/VoOsBZZc+9D21CxmLNzG7OVJ5OQXMrRjPI9e2ovhnRIIsAGIxpiTiKouEJG2FexfVOLtYsCmjTCmNFVXEtL6DGjSturni8Dw38Nbl7s5svuUXii7CjZ8Cu/9CqJbwsT3oUmbE2+rgbPkuppUlcVbDzBj4Tb+s34fQQHC2D4tuH5oO7o1b5jLehpjjI9dD3xW3k4RmQJMAWjdunVNxWRM7UteCamb4KKbTryNTue5HuZvn4TeV7jl0qvqh1nw4Y2Q2BeuetfNSGLKZcn1CUrPyef9lUnMXLKTLfsP0yQ8mJtHduSawW1oanNOG2OMV0RkJC65HlreMao6HVc2woABA2xZP3Py+OEdCGwE3cedeBsiMOxOmP1LWPsB9LqsauenboGPb4c2Q+Cqd9yc1aZCllxX0Q+7DvHm4h18vDqZ3PwiTm0Vw98u682Y3omEhVg9tTHGeEtEegMvAaNVNa224zGmTinMhzXvQZfREBZTvba6XgQJXV3vdY/xbrCjN4oK4aPfuLm1x79oibWXLLn2QnZeAXNXJTNzyU5+3J1OWHAgl/RtwdWD2tCzhZ9XPjLGmAZIRFoD7wPXqOqm2o7HmBqxe6Wbe7rftdDz0oqT3C3/hew0OPXK6l83IADOnArvT4aNn0K3i7w7b/ELbh7rS6ZD4+bVj+MkYcl1Jf75zU88/9UWMo8U0LlZJA+N7cG4vi1oHFrNEbfGGNOAicjbwAggXkSSgPuBYABV/SdwHxAH/MMzLWmBqg6onWiNqQG56fDuJEjf5abWW/QMnPMgdDy77ON/mAXhcdDxHN9cv8cl8PUjsOBv0HVM5dP6pWyCrx6GLhee2EwlJzFLriuQnVfA4//ewGltY5l6fhcGtGlic1MbY4wXVLXCpeRUdTIwuYbCMaZ2qcKnv3OLr0ya556/esjNF91uOJz7oBssWCznkFsApv+k6k+fVywwyC1RPvdm1yve6dzyjy0qdAMYg8NgzNO2pHkVeVl0c3LasDcTVbh+aDsGto21xNoYY4wxVbf6HTcV3oi7oc3p0PsXcPNyGPUY7P0Rpo+Ad69zKx4CrPsQCo/AqVf4No5Tr4ToVvDNX13CX55Ff4fdy+GCJyCqmW9jOAlYcl2BtckZAPSwumpjjDHGnIi0n1yvdZshcObvjm0PagSDb4TbfoBhv4dN/4bnBsK8O2Hl6xDfGRL7+TaWwGAYejskLYVtC8o+Zv8GmP+Iq8vuealvr3+S8GtyLSKjRGSjiGwRkbvL2N9aROaLyPcislpELiix7x7PeRtF5Hx/xlmedcnpxIQHkxhtU+sZY4wxpooK8mDOZAgIgvHTy55jOrQxnPVHuPV7N9Bx2cuwe4Wbk9ofd8z7TITIU1ztdWmFBa4cJCQCLnzKykFOkN+SaxEJBJ4HRgPdgQki0r3UYfcCs1W1L3Al8A/Pud0973sAo3ADXmp8nru1yRn0SGxs5SDGGGOMqbr5f3ELwVz8d7eyYUWiToExT8FNS2HEPTDQT0MSgkNhyK1uUOXOxcfvW/SMi/fCJyGyqX+ufxLwZ8/1acAWVd2qqnnALGBsqWMUKF7GMBpI9rweC8xS1SOqug3Y4mmvxuQXFrFhbybdbZVFY4wxxlTV1q9h4TNuUGL3i70/L76jq82u7tzWFek/CcLjj++93rcO5j/qFqzpOd5/1z4J+DO5bgHsKvE+ybOtpAeAiZ5pmuYBt1ThXERkiogsF5HlKSkpvoobgK0pWeQVFNEj0eqtjTHGGFMFWanw/v9BfCc4/5HajubnQiLg9JvcrCG7V7oFaz68EUKjXa+1qRZ/Jtdl1VKUHpo6AXhVVVsCFwBviEiAl+eiqtNVdYCqDkhISKh2wCWtTU4HoEei9VwbY4wxxkuq8NHNkHMALnul7q5qOHAyhMa4VRu/mwZ7VrmylIj42o6s3vPnPNdJQKsS71tyrOyj2PW4mmpU9X8iEgrEe3muX61NziA0OID2CZE1eVljjDHG1GdLX4RNn7lp9k7pVdvRlC+0sZut5OtH3UwlPS+F7qWrd82J8GfP9TKgk4i0E5EQ3ADFuaWO2QmcDSAi3YBQIMVz3JUi0khE2gGdgKV+jPVn1ian0+WUxgQG2GBGY4wx5qRVkAef/xGeH+xWWFzwBGz6AjKSfz5X9N418MW90Ok8GHRDrYRbJadNgZAoCGsCo8uYPcScEL/1XKtqgYjcDHwOBAKvqOpaEXkIWK6qc4HfAS+KyG9xZR+TVFWBtSIyG1gHFAA3qWqhv2ItI3bWJWcw5tTEmrqkMcYYY+qajD3w7rWwawm0PROSv4e1HxzbHx4HzXq6HupTesF3T7u65bH/qB/T2IXHwjUfuF7siLjajqbB8Ovy56o6DzdQseS2+0q8XgcMKefcvwB/8Wd85Uk6mENGboHVWxtjjDEnq+3fuVUT87Jc7XTxgiq56bBvreul3rsa9q1xpSCFR9z+ie9DpG/HgflVq4G1HUGD49fkur46ujKjzRRijDHGnFxUYfE/4Is/QWw7uHYuNO12bH9oNLQ5wz2KFRZA2hYoyIXEPjUfs6lTLLkuw7rkdAIEujSLqu1QjDHGGFNTjhyGuTe70o+uY2DcC65kojKBQdC0q//jM/WCJddlWJucQYeESMJCanxRSGOMMcbUhtTN8M5ESN0E5zwIQ26rH3XTps6x5LoM6/ZkMKhdbG2HYYwxxpiasP5j+OBGCAqBaz6E9sNrOyJTj1lyXcqBrDz2pOf6vt5aFYoK3a2j6spKA19MnhIY4t/lVY0xxpi6bvE/4d93QYv+cPnrEN2ytiMy9Zwl16UUr8zY3dczhcy9GVI2wvX/qd5tpqUvwrypvotr0jxoW+aELcYYY0zDVlToVihsNwyufg+CGtV2RKYBqDS5FpFmwCNAoqqOFpHuwOmq+rLfo6sFx2YK8WFyrQobP4PsNPfc9YITayc/101en9gP+l5dvZiKCuGz30PSUkuujTHGnJx2LIKs/dD/cUusjc9403P9KjAD+KPn/SbgHaDBJtctYsKICQ/xXaOpm1xiDbDgb9Bl9In1Xn//BhzeC5e+6L5lV9e3T0HKpuq3Y4wxxtRH6z6EoDDofH5tR2IaEG+WP49X1dlAEbiVF4EaWy2xpq1LTvd9SciOhe75jFsheSX89FXV2yjIg4XPQKtBbpUoX0joAqkbfdOWMcYYU58UFcK6udD5PAiJqO1oTAPiTXKdJSJxuOXJEZHBQLpfo6ol2XkFbE3NontzXyfXiyCyGZx1LzRu4Uo7qmr1O5C+C4bd6bupgRK6uJ5rVd+0Z4wxxtQXxSUhPS6p7UhMA+NNcn0HMBfoICILgdeBW/waVS1ZvycTVT/UW29f6FZyCmrk5s3cucht81ZhgRtw0bwPdDzHd7EldIG8TMhI9l2bxhhjTH2w9gNXEtLpvNqOxDQwlSbXqroSGA6cAfwf0ENVV/s7sNqwzjNTSI8WPpyG79AOyEyGNp5Bg/1+CRFNYcFfvW9j7ftwcJtve60B4ru455QNvmvTGGOMqeuKCmH9XFdrbSUhxscqTa5F5JfAVUB/oB8wwbOtwVm3J4OY8GASo0N91+iORe65zRnuOTgMzrgFtn4Nu5ZVfn5RkSsjadodupzgLCPlSfAk16k2qNEY41si8oqI7BeRNeXsFxF5VkS2iMhqEelX0zGak9iOhZCVAj3G1XYkpgHypixkYInHmcADwMV+jKnWrE3OoEdiY8SXvcM7FkJYE0jodmzbgF+5bd96UXu94WM36PDM30GAN/9cVRCR4OJIsUGNxhifexUYVcH+0UAnz2MK8EINxGSMs/ZDCA63khDjF96UhdxS4vFroC/gw3nq6ob8wiI27M30z2DG1mccnxg3ioTBN8Gmf8OeH8o/V9VN3RfX0T8DLkRcaYgl18YYH1PVBcCBCg4ZC7yuzmIgRkSa10x0pkEoyHN3d6uquCSkk80SYvzjRLpCs3E9DQ3KTymHySso8u2y5xl74MDWYyUhJQ2aAo2i3UDF8mz6HPb+6Om1DvRdXCXZdHzGmNrRAthV4n2SZ5sxlTuSCc+cCv+9v+rnHi0JsVlCjH94U3P9sYjM9Tw+ATYCH3nTuIiMEpGNnpq6u8vY/7SIrPI8NonIoRL7Ckvsm1uVD3Ui1u72w8qMxfNbl5Vch0a7BHvdXNhfxoDC4l7rmNbQ6xe+i6m0hC5ugZusVP9dwxhjfq6s+rsy5wUVkSkislxElqekpPg5LFMvLPmXmyxg8QuuE6sq1n5gJSHGr7zpuX4CeNLzeBQYpqo/S5RLE5FA4HlcXV133EDI7iWPUdXfqmofVe0D/B14v8TunOJ9qur3Gu91ezIIDQ6gfUKk7xrdsQhCIuGU3mXvH3Sj+x+8rN7rrV/D7uUw9LcQGOy7mEo7OmOI9V4bY2pUEtCqxPuWQJnzgqrqdFUdoKoDEhISaiQ4U4flpsOiv0Pr093fxy8f9v7cwgJY/7FnlpBw/8VoTmre1Fx/U+KxUFWTvGz7NGCLqm5V1TxgFq7GrjwTgLe9bNvn1ian0+WUxgQG+HIw4yK3omJgOavMR8TBwF/Bmvcg7afj9y34G0Q1hz5X+y6eshydMcSSa2PMz4lIZxH5snjWDxHpLSL3+qDpucAvPbOGDAbSVXWPD9o1Dd2Sf0HuIRj1KJx+s5uuNmmFd+cWl4R0t1lCjP+Um1yLSKaIZJTxyBSRDC/a9rqeTkTaAO2AkuuCh3puAy4WEb/+X6CqrPPMFOIzWWmQsr7skpCSTr8FAkPgu6ePbdu+0P0CGHKbW3jGn6JbQnCE9VwbY8rzInAPkA/gWefgyspOEpG3gf8BXUQkSUSuF5EbROQGzyHzgK3AFs81fuOP4E0Dk3MIFj0HXS6ExL4w5FY389V//uTdasPrbJYQ43/ldKmCqkZVs22v6+lwv6jfU9XCEttaq2qyiLQHvhKRH1X1uO5dEZmCm8KJ1q1bn3CgSQdzyMgt8G1yvfN/7rl48ZjyRDWDftfC8pdh+F0Q08pN0ReR4Lb7mwgkdLbk2hhTnnBVXVpqitKCyk5S1QmV7FfgpmrGZk42i1+AI+kwwlOd2ijKvf70d24Gri6jyz+3sMCNc7KSEONnXs8WIiJNRaR18cOLU7yup8Ml18eVhKhqsud5K/A1bgpASh3jkzq8tcUrM/pyppAdiyAoFFp4sS7CkFsBgYXPuFtbP33lbnXV1P/8Nh2fMaZ8qSLSAU/niIhcBlj5hql5OQdh8T+g20XQvMRYpn7Xuilr/3O/S6DLs2MhZKfaLCHG77yZLeRiEdkMbAO+AbYDn3nR9jKgk4i0E5EQXAL9s1k/RKQL0AR3+7B4WxMRaeR5HQ8MAdZ5cc0Tsi45gwCBLs2q21lfwo6F0HKgd2Ud0S2hz1Ww8nX44o8QGgMDr/ddLJVJ6OxGXed6U+1jjDnJ3AT8C+gqIruB24Ebazckc1L63/NwJANG3HP89sBgOOcBN3Zo1Zvln188S0jHc/0ZpTHll4WU8DAwGPivqvYVkZG4wYcVUtUCEbkZ+BwIBF5R1bUi8hCwXFWLE+0JwCzPLcJi3YB/iUgR7gvAY6rq2+S6MB+2fgOAbNnIFU2OELajjERYgJanQWgVSkZyM2Dvahh2p/fnDP0tfP+mKycZ8Qd3q6umJHR1z6mboWX/mruuMabO89w9PEdEIoAAVc2s7ZjMSSj7gCsJ6T4OmvX4+f6uY6DVYJj/iJu+tvTiMEdnCRllJSHG77xJrvNVNU1EAkQkQFXni8jj3jSuqvNwg1ZKbruv1PsHyjhvEdDLm2ucsLzDMPNSAH5bvG1mOcf2GA+/mOF927uWghZVPpixpNh2cOoEt9z5oCnen+cLR6fj22DJtTHmOCJyX6n3AKjqQ7USkDk5LXoW8rKO1VqXJgLnPQwvn+sGPI646/j9R0tCbJYQ43/eJNeHRCQSWADMFJH9eDGYpc4LiYLr/0t6Th6TZixj0hltGdunjMlMfngLls+AkX+AeC8XptzxHQQEubKQqrjwSTj7PghrUrXzqqtJWzdjiU3HZ4z5uawSr0OBMcD6WorFnIyyUmHJdOg5Hpp2K/+4VqdBt4vd+KUB10Fk02P71n7gZsaykhBTA7xJrscCubgO3quBaKD+91gEBkGrgfywKYXv9RBTuw2CVvE/Py62Hax62y30csk/vWt7xyI3RVDp21KVCQ51j5oWGOQGg9igRmNMKap63CpXIvIEZYyfMcZvFj4DBTkwvNL161zt9cZ58PVjMOYpt80WjjE1rKJ5rp8TkTNUNUtVC1W1QFVfU9VnVTWtJoP0p3V7Kln2PCIeBvwKVs+GA9sqbzAvG3avrFpJSF0Qb9PxGWO8Eg60r+0gzEni8H5Y9hL0vMwNvq9MXAf3N3vFq24cEbi7yVYSYmpQRbOFbAaeFJHtIvK4iPSpqaBq0trkDFrEhBETHlL+QWfcAgGBsHBa5Q3uXg5F+ZXPb13XJHSFQzsgP6e2IzHG1CEi8qOIrPY81gIbgWdqOy5zklj4DBTkunUgvDXs925WkP8+4N6v/dBKQkyNKje5VtVnVPV0YDhwAJghIutF5D4R8eLrY/2wNjmd7pUtHtO4OfS9Br6fCem7Kz52xyJAoPVgn8VYIxI6u0GYaVtqOxJjTN0yBrjI8zgPSFTV52o3JHNSyNzreq17XwnxHb0/LzIBht4GGz6B7d/Bels4xtSsSue5VtUdqvq4qvYFrgIuoYEMZsk6UsC21Cy6N/dimr2htwPqvkVXZMdCOKUXhPpwQZqaUDwdn5WGGGMAEYkVkVggs8QjB2js2W6Mf303zU2bO7wK09oWG3wTRDWHdydBdpotHGNqlDeLyASLyEUiMhO3eMwm4FK/R1YDNuzNRLWCeuuSYlrDqVfCytcgc1/ZxxTkwa5l9a8kBNyARgmw5NoYU2wFsNzzXPqxvBbjMieDjGRY/gr0mQCxJ1DiHxLuZvnKSnElIZ2sJMTUnHJnCxGRc3ELvFwILAVmAVNUNau8c+qbdcXLnrfwspd56B2w6i3433NuPs3S9qxyI5rr22BGcCtJNmlr0/EZYwBQ1Xa1HYM5iX33NGhh1RZjK63P1S5BP6UXBIf5LjZjKlHRVHx/AN4CpqrqgRqKp0atTc4gJjyYxGgvp7+L6wA9L4VlL7sVFcNL3RndsdA918fkGlxpSMqm2o7CGFPHiEgToBNunmsAVHVB7UVkGrSfvnKzffS52nX6nKiAQJj8pbsra0wNqmhA40hVfbGhJtbgkuseiY2PrjjmlTOnQn6WW4a1tO0L3WqHEWXMl10fxHd2AxoL6/8aQcYY3xCRybhFxD4HHvQ8P1CbMZkGbMM8eOsK9/fo7Pur315AoFu90ZgadNJ+ncsvLGLj3kzvBjOW1LSrWwFqyb8gN/3Y9qJC2Lm4/vZag+u5LsqHg17M522MOVncBgwEdqjqSKAvkFK7IZkG6cf34J2Jrozj2o8hIq62IzLmhJy0yfWh7Hz6t2lC/zYnsNT4sKlwJB2WTj+2be+PkJdZPwczFiueoD9lQ+3GYYypS3JVNRdARBqp6gagSy3HZBqalW/AnMluGttffvTzsktj6hFvZgu52VNv16AkRDXi7SmDGdWzedVPbn4qdDof/vcPOHLYbduxyD3X557r+OLk2gY1GmOOShKRGOBD4D8i8hGQXMsxmYZkyb9g7s3Q4Sy4+j1oFFXbERlTLd70XJ8CLBOR2SIySqpUoNyADbsTcg7Aihnu/Y6FbuBFdItaDataGkVB45aQaoMajTGOql6iqodU9QHgT8DLgK0jbXzj2yfhs99D1zEw4W1b6MU0CN4sInMvbpT4y8AkYLOIPCIiHfwcW93WaiC0HwELn4W8bNdzXZ9LQooldLayEGMMIvKpiFwtIhHF21T1G1Wdq6r/z96dx0dZXY8f/5zsO0lIWEMSwLDLGkEWUbQouLAoLihVtK5fqVWrLVZr1Wpr1Z/a1hUVXKpg1aq4oKKiIDsooIQdEghhSdhCgIQs9/fHnQmTkGUSMpmZ5Lxfr3lN5pnneeYwJE9O7px77nFvxqZ81J618PHt9vdizio7F6k6xsA3j9jb6ZfD5a/blrBKNQE1teIrZ4wxIrIb2A2UAHHA+yIy1xjzB08G6NOG3wuvXwRfPWBHsf25JMQpsZttgVRWBgHNtiRfKQXTgKuAZ0XkW2Am8Lkm1qpKezLgjUtsqWRpkd0WFgupw6Dj2dDxLPv7RcQm1l/cB0tfhP7XwsXP2q4eSjURtSbXInIHcB2QB7wK3GuMKRaRAGAT0HyT65ShkDwYVrzmeNwEkuuELlB8FA7tgLgUb0ejlPISY8zHwMciEg6Mwf4eeElEPgdmGmPmejVA5Tv2rreJdWAI/N9iCI6AzAWw7XvYNh/Wf2r3i2xlk+yyEsj4GAbdBqP+rq3yVJPjzsh1AnCpMSbLdaMxpkxELq7pQBEZBfwTCAReNcY8Xun5Z4ARjocRQCtjTKzjueuABxzPPWqMecONWBuXiB29/s+lEN0W4prAgmaJjiYAeRvdT643zYWEtFNr9g+wdx0U5kPyoFM7j1KqwRhjjgHvAu+KSG/gDWyirUONyi489sYlduT5uk/sYmsAva+wN4ADmbBtgU20t82Hgt12zYhzH9DEWjVJ7iTXnwPlC8mISDTQwxiz1BizrrqDRCQQeB4YCWRjJ0XONsZkOPcxxtzlsv9veW2vPQAAIABJREFUsf1TEZF44C9AOmCAlY5jD9TlH9coOp8LnUac+LjL3yV2s/e5GyBtZO3779sC71wBPS+FCa+d2mt/9nt7vt+vbxrvpVJNgIi0Bq7Aloi0Bd4DrvdqUMo35G22iTXAdZ/aQZaqxKXaW/9f25KQosMQVsc1JpTyI+4k1y8C/V0eH6liW1UGApuNMVsBRGQWMBbIqGb/idiEGuACYK5zdUgRmQuMwtb8+RYRuPYjb0fRcCLiISLB/UmNPzwNpsxO6DSm/klxcSFkL4fS47B/64nRD6WUV4jITdjrclfgf8AfjDELvRuV8hn7tsAbF9sSj8mfnlgnoTYimlirJs+dGWtijDHOB8aYMtxLytsDO1weZzu2nfwCIilAR+DbuhwrIjeLyAoRWZGbqwuGNZjEbu614zu4HVbPgqjWcDjHfvRXXztX2sQaTvQMV0p50xDgcaCDMea39UmsHe1bN4jIZhGZWsXzySIyT0R+EpE1InJhQwSuPGz/NjtiXXocrpsNrbp7OyKlfIo7yfVWEblDRIIdt98BW904rqohTFPFNrAfN75vjHH27XHrWGPMNGNMujEmPTEx0Y2QlFuc7fhMdf9dDgv/CQiMfd4+PpWk2HlsaAtNrpXyAcaY640xXzkGVOrMpTRwNNADmCgiPSrt9gDwX2NMP+zvgRdOJWbVCA5k2cS6+KhdSbF1T29HpJTPcSe5vhU7grETO4I8CLjZjeOygQ4uj5OoflWvq6hY8lGXY1VDS+gKhYegYG/1++TvssvV9r0aOp8H4XGnmFwvhFY9odNw+7VSyt+VlwY62vc5SwNdGcBZI9ACvc57x4Yv4KP/g3l/g1XvQNZie42vPMBycIctBSnKt4l1m9O9E69SPq7W8g5jzF5s8ltXy4E0EemITcyvAq6uvJOIdMX2zV7ssvlL4G8uy66fD9xXjxhUfTg7huSuh+jWVe+z6N+21m7YXbYfdvKQ+ifFpcWwY5lN1Ft2hnWfwKFsaJFUv/MppXxBVeV9lVsBPQR85ZjQHgn8qqoTicjNOAZ1kpOTGzzQZm31u/DRrRASDccP2zk0TkHhtmtUXEc7IXHjHDh2CK77GNr28VrISvk6d/pchwG/AXoCYc7txpgbajrOGFMiIlOwiXIgMN0Ys1ZEHgFWGGNmO3adCMyqVNe9X0T+ik3QAR5xTm5UjcC1HV+ns09+/kgerJhu2yzFO9oPpgyBDZ9Bfg7EtKvb6+1aA8VH7DmcExmzFkPvy+v/b1BKNQjHarzZxpgiETkH6A28aYw5WNuhVWyrXGs2EXjdGPP/RGQw8JaI9KpcimKMmYZd1Ib09PRa6tWU2356266o2PEsmDgLAoLtXJoDmXBgm73f77jf9r3tX/3rD6FdPy8HrpRvc2di4lvAemwHj0eAa4BqW/C5MsZ8jm3l57rtwUqPH6rm2OnAdHdeRzWw6LYQGmPb8VVl8fNQUgjD7j6xzbmATtYiOH1C3V7POeKdMhQiExx11ws1uVbKN3wApIvIacBrwGzgHaC2yYfulPf9BtsJCmPMYsdgTgJQQ02aahArX4dPfmfbyV71DgSH2+0Jp9lbZcbYUW1dSVGpWrlTc32aMebPwBHHQi4XAVpo1ZSJ2JUaq2rHd+wALHsFeo6r2HqpTW/7sWJ96q6zFkLL02wJSkAgJJ+pdddK+Y4yY0wJMB541rE+QVs3jisvDRSREGxp4OxK+2wHzgMQke7YT0e19ZOnLXvFJtZp58NVM08k1jUR0cRaKTe5k1wXO+4Pikgv7KSTVI9FpHxDYteq2/EtfdnW5Z11T8XtgUF2ZcW6JtdlpbYExHXp+JQh9rUL9HesUj6gWEQmYldldKxjTXBtBzkScmdp4DpsV5C1IvKIiIxx7PZ74CYRWY2d1D7ZtUSwQRTshc1fN+gp/dqSF+Hze6DrhXDlfyA4rPZjlFJ14k5yPc0xsfAB7KhDBvAPj0alvC+xKxTssSPVToX59sLc9SJo0+vkY1KGQO46OLLP/dfZmwFFh2xJSPl5HF9v15Z8SvmA64HBwGPGmG2OSer/cedAY8znxpguxpjOxpjHHNsedM65McZkGGOGGmP6GGP6GmO+avDo5/wR3v015G1q8FP7nYX/gi+mQvdL4PI3ICjU2xEp1STVmFyLSACQb4w5YIyZb4zpZIxpZYx5uZHiU96S4OwY4jJ6veI1KDwIw39f9TH1SYqdI92uI9dt+9iJM9rvWimvcyTAdxhjZjoGWqKNMY97Oy63XfA3CAqD96+HkiJvR+M985+CuX+GnuNhwgwICvF2REo1WTUm144Z21MaKRblS8o7hjgmNR4/Couesz2t2w+o+ph2/ewvsbokxVkLoUUHiHVprxUUAklnaN21Uj5ARL4TkRgRiQdWAzNE5Glvx+WuQ0EJbB32FOz+Gb5+yNvhND5j4LvH4du/wumXw6WvQmCtVT1KqVPgTlnIXBG5R0Q6iEi88+bxyJR3xSbbRNnZMeTHN+BoHgy/t/pjgkLrlhQbYxNx11Frp5ShsPsXOFZbty+llIe1MMbkA5cCM4wxA6imH7Uv+svsX7j82xjMwFtgyQuw8Utvh+R5pSW2xenyV+HdSfDd36HP1TD+ZTs/RinlUe78lDn7Wd/uss0AnRo+HOUzAgKhZZpNrosL7VLnKcMgZXDNx6UMgflP2hUew1rUvO++zXAkt5rkeghgYMdS6HJBvf8ZSqlTFiQibYErgPu9HUxdDe+SyEercsjo+Xt6Zi2Cj26DWxdCjDsNT/zEsQOQvcJeL3csg50r4XiBfS6yFQz9HZz3kF3wSynlce6s0NixMQJRPiixq71Qr3obDu+C8S/VfkzKENsLdccySBtZ877l/a2HnfxcUrpd0CBroSbXSnnXI9iOHwuNMctFpBPgN7MDz0pLBOC7rYfpOWE6TDsbPrwZfv2Rf7eWMwZ+eAbWvHuibaoEQOte0GcidBhob7Epto2eUqrRuLNC47VVbTfGvNnw4SifktgVfnkfFvw/W+7RsYrVGitLOgMCgmxSXFtynbnQjqo4V2V0FRxua7szte5aKW8yxrwHvOfyeCtwmfciqpvE6FB6tI3h+4253D5iMIx+AmZPsZ/GnXV37SfwVctegW8etiV0Ix6wiXT7ARAa5e3IlGr23CkLOcPl6zBsw/8fAU2umzrnpMb8nXDxM+6NfoREQrv+tU9qNMYm4ClDqj9vyhBY9C8oKtBfGEp5iYgkAf8GhmJLAn8AfmeMyfZqYHUwvEsiry7YyuHCYqL7TYIt38K3j0LqWdDhjNpP4Gu2fGtb6nUZbVdX1HIPpXxKrT+RxpjfutxuAvoB2sOnOXC242vT267k5a6UIbDzR9thpDoHt9uk3bW/dWWpQ6GsBLKXu//a7jq8Bz7/g+3dfSrKSmHuX2DP2oaJqzlb8pJ+UuGbZmDXOGgHtAc+cWzzG8O7JFBSZli8ZZ/9Y/7iZ6BFe/jgBjs/xJ/kbYL/TobEbnDZK5pYK+WD6vNTeRRIa+hAlA9qeRp0u9j2ia1LzV7KUCgrhp0rqt+nqv7WlXUYZGsIPdHv+vt/wLKX7YqTp2LdbFj4rB1FUvVXdBi+/JOtIVW+JtEYM8MYU+K4vQ4kejuoukhPiSciJJD5mxyrvobHwmWvwaGd8Mmd9pM0f3DsAMy8ynb8mDgTQqO9HZFSqgq1Jtci8omIzHbcPgU2AB97PjTldYFBcNXb0PGsuh2XPAiQmpPirIW2m0irHtXvExptF5Rp6OQ6Pwd+egskEJY8bxO7+igrswszSCBsmw/blzZsnM3JjqVgSmH7EttGTPmSPBGZJCKBjtskoA7LsHpfSFAAgzu1ZP7GvBMbOwyEEX+Ctf+zk7arU1ZqF9Na856t0/bWp1SlJfDeZDiQZZctj0vxThxKqVq5U3P9lMvXJUCWP9XaKS8IawFtTq+533XWIkgeUvtHmilD7cSdkqKGW6p30b/tL8xLp8EHv4EV022rqrra+AXs+QUuehrmPWZbEE56v2FibG6cf0AdPwx7frYLEilfcQPwHPAMtuZ6EXZJdL8yvEsi36zfS2beEVITIu3GYXfB1u/g83vtJ2WxKbbzxq7VsHuN4/5nKHYpcZv7ICR2h9Mvg14TIL6RGmp9+Scb65jnav7ETynlde6UhWwHlhpjvjfGLAT2iUiqR6NS/i9lKOxYDiXHT37u8G7Yv8W9XxApQ6C0yNZwN4SCXFgxA3pfCadPgE4jbLJdfKxu5zHGJtNxqdD/Ohh8O2yeCzk/NUyczU3WohOrdOqy9z7FGLPdGDPGGJNojGlljBmHXVDGr5zdxVaylJeGgG3Fd+k0u2DWq+fB39vDy2fZbiKr3rFlaf2vhbEv2N7Yd6+HC5+yAwjfPgr/6guvnAdLXrTXNU9ZMd2WsQ2eAv1/7bnXUUo1CHdGrt8DXLOgUsc2P5xirRpNyhBY+qJNNpMHVXzOmTyl1jCZ0SnZsWhN1sLaF7Bxx5LnoaTwRAuus/8AM0bDj2/CoFvcP8+WbyHnR7jkX7Z85oyb7EfG85+ypTTKfcXH7KIXg26BdZ/a74/Bt9d+nPKmu4FnvR1EXaQmRJIcH8H8jblcOzj1xBMx7WDCdLt6Y6vudgJ3274Q36nqT9YG3mRvB7fDL/+z7Uq/mGpHllOHQfcxEBgCRfm25KzQcV906MTj4mO2S0mvCfaYmvptb5tvR9ZPGwkjH2nw90Up1fDcSa6DjDHlw4/GmOMi4la3EBEZBfwTCAReNcY8XsU+VwAPYT9uXG2MudqxvRT42bHbdmPMGHdeU/kI56h01sKqk+vgSGjTp/bzRMTbuuyshcA9pxbT0f22xKTneEhIOxFnylD44VkYMNm90hPnqHVMe7tYA0BYDAy61U6U3LMWWvc8tVibk50rofS4/X84egA2fG7r2bULgi/zy1VJhndJ4H8/7uR4SRkhQS7fX51H2FtdxCbDsDvtLXcD/Py+TbQ/r3SdCo2x80ec9xHxdp7Gzx/YP+qj2thr0ukTbJ9q18nj+7fCf6+F+M4w4TX/XvRGqWbEneQ6V0TGGGNmA4jIWCCvlmMQkUDgeWAkkA0sF5HZxpgMl33SgPuAocaYAyLSyuUUx4wxfevwb1G+JDLBtorKWnTyQg3OhDvQnW8/bAK8aqad0OPuMVVZ+rJdEnh4pV9+w++Bt8bbj4HT3SglzVoI2xfD6CchyOXvzEG3wuLn7aI7E6bXP87mJmsRIJB8pu2GsOo/kLfBjiIqX+Un7TUqGp6WyH+WbGdF1n6GdE5ouBMndoVz77cTJA9kQmCwTaRDoqv/I/H4Udj0pU3KV7xmP+mLS4VejlruFu3hnavsvlfPsqUoSim/4M7Q0K3An0Rku4hsB/4IuPP5+UBgszFmq2PkexYwttI+NwHPG2MOABhj9rofuvJ5KUNs94ey0hPbju6HvRl1m5CTMgSKj8Du1fWPpTDf/vLqdvHJo8qdRtgRox+ehtLi2s81/0m7smTl2seIeDjjRvtRcZ7frA7tfVkL7ZLN4XEVP/FQXiUih0Ukv4rbYWzPa78zuHNLggKkYteQhiRiJzi2SLLJcE2fvoRE2BHrq96Gezfbuu74TvZTtBcHw7On27kpV7xptyul/IY7i8hsMcacCfQAehpjhhhjNrtx7vbADpfH2Y5trroAXURkoYgscZSROIWJyArH9nFuvJ7yNSlDbfeH3T+f2LZ98Ynn3JXsTLhOYaLb8lftYhFn/f7k50Rg+B9sDeXPtXT72LHcztgfeoddor2ywVPs5Cjt1+ye0mLYsexEUh2bYsttdFKj1xljoo0xMVXcoo0xp/ARkvdEhwXTPyWO+Rtza9+5MYW1gH7XwK8/hN9vsJMm2/WznUE6Dvd2dEqpOnKnz/XfRCTWGFNgjDksInEi8qgb566qJq/yR4lB2AVpzgEmAq+KSKzjuWRjTDpwNfCsiHSuIrabHQn4itxcH7tYKpfJiC6JUtYiCAy1S6S7K6atHbmpb8J1/Agsfg5O+xW0r+Z1u1wArU+3JR2uI+2VLXgKwuNhQDXlI1GJtnZ79Sz78bCqWc4q2+bMmVyL2K8zF/rPwh7Kr5zdJZGMXfnkHi7ydihVi0q0Eyav/Rj6TvR2NEqpenCnLGS0Meag84GjhONCN47LBjq4PE4CcqrY52NjTLExZht2gZo0x+vkOO63At9hl12vwBgzzRiTboxJT0z0qwXDmocW7W0NoetH/FkLISkdgsPqdq6UoTa5LiurexwrX4ej+2D4vdXvI2Jrr/dtgoyPqt5n12rb23rw/0FoVPXnGnqHnXj0g181U/AO5/eGa5lQyhAo2G0ncynVwIan2d8VCzbpgIxSyjPcSa4DRaS8hYKIhAPurOaxHEgTkY6O7iJXAbMr7fMRMMJx3gRsmchWx+h4qMv2oUAGyv84k2JjbBuqXavrtwBCylAoPAi56+p2XHEhLPwXpJ5lJ8zVpPsYSOhq2+lVlcTPfwpCW8DAm2s+T0w76DfJrvp2aGfd4m1ushZByzSIcpnL7CwZ0tIQ5QE928XQMjLE90pDlFJNhjvJ9X+Ab0TkNyJyAzAXeLO2g4wxJcAU4EtgHfBfY8xaEXlERJxt9b7ELkqTAcwD7jXG7AO6AytEZLVj++OuXUaUH0kZAsf221ZVO5aCKatbvbXreaDuCdeq/9hR0JpGrZ0CAmxN9t4M2Din4nN718G62TDoZvdm7Q+905aXLPp33eJtTsocy51X/mMroQtEtNTkWnlEQIAwLC2BBZvyKCvT0iOlVMNzZ0LjE8Cj2IS3J/BXY8w/3Dm5MeZzY0wXY0xnY8xjjm0POtv6GetuY0wPY8zpxphZju2LHI/7OO5fq+8/UHlZ+SjkDzZZCgiCDgPrfp7YZIhJgswf3D+m5LgtzUga6P6koF6XQVxH2xHEteZ3wdO2N/eZ/+feeeJSoM9VsHIGFGgTnCrtWWsX1qj8x5az7lo7higPGZ6WyL4jx8nYle/tUJRSTZBbqzQYY74wxtxjjPk9UCAiz3s4LtVUxKVCdDubWGcutCufhUTW/TzlCdci9ye6rXkXDu2wo9bi5poXgUG2L3fOT7D5G7tt3xa7OMQZv7Ht9tw17G67OMri59w/pjmpaaXOlKFwMAsOZTduTKpZOKuL7XH9vZaGKKU8wK3kWkT6isg/RCQTO4q93qNRqabDmRRvm29X4qtPvbVTyhA4stcmu7UpLbF9q9v2gbSRdXud3lfZUfL5T9hE/oen7XLGg6fU7TwJp0HPS2H5a7a/t6ooa6H9RKJF0snPlZcBLW7cmFSDEpFRIrJBRDaLyNRq9rlCRDJEZK2IvNMYcbWKDqNH2xhNrpVSHlFtci0iXUTkQRFZBzyH7ewhxpgRxhgtJFXuSxkCR3KhrLh+9dbl53GWmLhRLrD2Q9ttoi6j1k5BIXZJ4x1LYfVM21av/3UQ3bruMZ/1e7sq5NKX6n5sU2aMHbmu7vuhdS87eVRLQ/yWyyq9o7HrJEwUkR6V9nFdpbcncGdjxTe8SyI/Zh3gcKEbC0cppVQd1DRyvR44D7jEGDPMkVDX0ABYqWqUJ1Bilz2vr4Q0iEiofaJbWZntR92qB3S9qH6v1W8SRLWGj28HxLbXq4/WPeyqkEtesovYKCtvIxzNq/6TjIBA291Fk2t/5tOr9A7vkkBJmWHxln2N9ZJKqWaiplW2LsO2z5snIl9gL4x1HAJUCkjsars/RLezS1zXl7PEZO3/YMeS6vcrK7W11pe9VvPywzUJDochd8BX90P/a6suXXDX8Hth/afw3BlVr+roasBkGHZX/V8LYN0nsOa/cPnrNkn1ReX9rWv4JCNlCGz6Egpy7cIayt9UtUpv5b+uuwCIyEIgEHjIGPNF5ROJyM3AzQDJyckNElx6SjwRIYHM35TL+T3bNMg5lVIKakiujTEfAh+KSCQwDrgLaC0iLwIfGmO+aqQYlb8TgdFPuNfCrjZDfwfBEZy82GclUa2g5/hTe630G2yNt7sdQqrTri+M/Cvs+aXm/bIW2+XXTzW5XvuhbRuY8ZHtfuKLshbZTwbiO1W/jzPx3r4IelQe8FR+oK6r9CYBC0Skl+vCZWAXDAOmAaSnpzdI/7yQoAAGd2rJ/I15DXE6pZQqV9PINQDGmCPA28DbIhIPXA5MBTS5Vu47fULDnCcp3d4aQ0gEjHykYc7lTlnJVw/A0ml25P1URpxzN9r7+U9Bj/H1H733FGNs55iUITXXw7ftY/+QytLk2k+5u0rvEmNMMbBNRJyr9C5vjACHd0nkm/V7ycw7QmpCPboYKaVUFer0W9cYs98Y87Ix5lxPBaRUs5XQFUqL4EBm/c9RVmrrmWNTql4MxxcczILDObVPbg0KgaQztO7af9V7ld7GCnB4F1tuNF+XQldKNSAfG9JSqhlL7Gbv8zbW/xwHs2yCPuwu22P8+yfc7wveWJwTUt3pHJMyFHb/AscO1r6v8imnuEpvo0htGUGH+HBdCl0p1aA0uVbKVyR2sfe5p9BGPneDvW/d0y5is2vVicVwfEXWQjux1fnHRE1ShgDGtkVUfqe+q/Q2FhFheFoii7bs43hJWWO+tFKqCdPkWilfEdYCotueqJmuD2dyndAF+kysuBiOr8haBMlD3KsFT0qHgGAtDVEec3aXRI4eL2VFli70pJRqGJpcK+VLErpA3ob6H5+7AaLaQHhsxcVwMn9ouBhPRf4uu7iPuyt1BodD+wF2AqRSHjC4c0uCAkS7hiilGowm10r5ksSuduS6viPNeRtOlJfAicVw5j/ZMPGdqvL+1m4m1859d62CogLPxKSateiwYPqnxGndtVKqwWhyrZQvSewKxw9DfuWOZW4wxibmrrXMweEw5Lew7XvYsazh4qyvrEUQEgVtert/TMpQKCuB7EbpzqaaoZHdW5OxK59n5m7E+FIJlVLKL2lyrZQvSehq7+szqTE/xybmCV0qbh9wPYTH+8boddYi6DAIAmttsX9C8iCQgNqXvVeqnq4fmspl/ZP45zeb+MvstZSVaYKtlKo/Ta6V8iWn0o7PWatduQtHaBQMvh02fQU5q04tvlNxZB/krqtbSQhAaLRdUEaTa+UhQYEBPDmhNzed1ZE3F2fxu3dXafcQpVS9aXKtlC+JTLBt6nLrManReUxi15OfG3gThLaABU+dWnynYvtie586rO7Hpgy1ZSElRQ0bk1IOAQHC/Rf1YOrobnyyOocb31zB0eMl3g5LKeWHPJpci8goEdkgIptFZGo1+1whIhkislZE3nHZfp2IbHLcrvNknEr5DBFbGlLf5DosFiITT34urAUMugXWfQJ71516nPWRtQiCwqBdv7ofmzLELo6z88eGj0spF7ee3Zl/XHY6P2zK5ZpXl3Lw6HFvh6SU8jMeS65FJBB4HhgN9AAmikiPSvukAfcBQ40xPYE7Hdvjgb8Ag4CBwF9EJM5TsSrlUxK71q8dX55jMqNI1c+feRsER8J8L41eZy20y5kHhdb92OTBJ86hlIddeUYyL1wzgLU787n8pcXsPlTo7ZCUUn7EkyPXA4HNxpitxpjjwCxgbKV9bgKeN8YcADDG7HVsvwCYa4zZ73huLjDKg7Eq5TsSu8LRfXCkjn13c9dXbMNXWUQ8nPEbWPs/yNt8ajHWVeEh2L2m7vXWThHx0KqHJteq0Yzq1YbXbziDXYcKuezFRWzN1VaQSin3eDK5bg/scHmc7djmqgvQRUQWisgSERlVh2MRkZtFZIWIrMjN1R6lqolw1kzXpTTkSJ5NyGtbUnzIbyEwBH54pv7x1ceOZWDK6p9cgz12+1Io1TpY1TiGdE5g1s1nUlhcyuUvLebn7EPeDkkp5Qc8mVxX9dl05f5GQUAacA4wEXhVRGLdPBZjzDRjTLoxJj0xsYo6U6X8kbMdX11KQ8qXPa9iMqOrqFYwYDKsmQUHsuoVXr1kLYSAIFsWUl8pQ6D4COxe3XBxKVWLXu1b8N6tgwkLDuTKaYt56ssN7D+iddhKqerVodlsnWUDHVweJwGVV8bIBpYYY4qBbSKyAZtsZ2MTbtdjv/NYpEr5khZJtja6LiPX5W34aigLcRpyByx/DX54Gkb+tX4x1lXmD3YiY0hk/c+R7Bj13jIPWqY1TFzuCImCAG2s1Jx1Sozig9uG8Mina3n+u81MX7iNSWemcONZHWkVHebt8JRSPsaTyfVyIE1EOgI7gauAqyvt8xF2xPp1EUnAlolsBbYAf3OZxHg+duKjUk2fiE2S65Jc5260CXlMUu37tmgP/a6Bla/bW2MZ+rtTOz6mLcR3hm//am+N5fcbILpN472e8kltWoTxwjUD2LTnMM/P28yrC7byxqJMJg5M5ubhnWgXG+7tEJVSPsJjybUxpkREpgBfAoHAdGPMWhF5BFhhjJnteO58EckASoF7jTH7AETkr9gEHeARY8x+T8WqlM9J6Arb5ru/f+56SEhzf4T1vL9AYne7rHhjCAiEXhNO/TyXvnKiX3ZjCYlq3NdTPi2tdTTPXtWPO3/VhRe+28x/lmTx9tIsJgxI4razTyO5ZYS3Q1RKeZkY0zSWeU1PTzcrVqzwdhhKNYwFT8M3D8PUHRAWU/v+T/eA1LPg0pc9H5uqlYisNMakezuOpsjXrvXZB47y8vdbeXfFDkrLDGP7tOOm4Z3o3taNn1ullN+q6TqvhYRK+SJnx5C8TbXvW5gP+Tvdq7dWSjWopLgI/jquFwv+MILrh6Qy55fdjP7nAq5+ZQnfrNtDWVnTGMBSSrlPk2ulfJGz60fu+tr3dSbgtXUKUUp5TOuYMB64uAeL7zuXP47qxtbcI/zmjRWc9/T3vLk4kyNF2kJSqeZCk2ulfFFcqu1H7U47PmcCXluPa6WUx8VGhHDbOZ1Z8McR/GtiP2LCg3nw47UM/vs3/P3zdew8eMzbISqlPMyT3UKUUvUVGAQtT7NdQGqTt8FhppjDAAAgAElEQVQm4nGpHg9LKeWe4MAAxvRpxyW92/Lj9oNM/2EbryzYyqs/bGNUrzacdVoCaa2jSWsdRUxYsLfDVUo1IE2ulfJViV0hZ1Xt++VutIl4oP44K+VrRIQBKXEMSIkj+8BR3lycxaxl2/lsza7yfdq2CCOtdTRdWkXRxZFwp7WOJipUf6aV8kf6k6uUr0roChkfQ/ExCK6hh27uemjbp/HiUkrVS1JcBH+6sDtTR3Uj+8AxNu45zMa9h9m0p4CNew7z1tZ9FJWUle/ft0Ms4/u156LebUmICvVi5EqputDkWilfldgFTBns2wxtTq96n+JjcDALel/RuLEppeotIEBIbhlBcssIftWjdfn20jLD9v1H2bjnMOt25fPFL7v5y+y1PPJpBmelJTCub3tG9mhNpI5oK+XTmvRPaHFxMdnZ2RQWFno7FOVjwsLCSEpKIjjYh2sdnRMUczdUn1zv22wT8ETtFKJ8j4iMAv6JXUjsVWPM49XsNwF4DzjDGOM7TawbWWCA0DEhko4JkVzQsw13/qoLG3Yf5qNVO5m9Koc7311FeHAg5/dszbi+7RmWlkBwoPYlUMrXNOnkOjs7m+joaFJTUxERb4ejfIQxhn379pGdnU3Hjh29HU71Wp4GEgB5NUxqdC6Rrm34lI8RkUDgeWAkkA0sF5HZxpiMSvtFA3cASxs/St/XtU00fxzVjXvP78qKrAN8tGonn/+8i49X5RAfGcKIrq04IzWO9NQ4OidG6e86pXxAk06uCwsLNbFWJxERWrZsSW5urrdDqVlQKMR1rLnXdd5Gm4C3PK3x4lLKPQOBzcaYrQAiMgsYC2RU2u+vwBPAPY0bnn8JCBAGdoxnYMd4HrqkJ99vzGX26hy+27CXD37MBiAuIpgBKfGkp8ZxRmocvdq3IDQo0MuRK9X8NOnkGtDEWlXJb74vErvW3I4vd71twRcc1mghKeWm9sAOl8fZwCDXHUSkH9DBGPOpiGhy7aaQoABG9mjNyB6tMcawLe8IK7IOsCJzPysyD/D1uj3l+/VJakHvpFg6J0bRKTGSzolRJESF+M81UCk/1OSTa2/at28f5513HgC7d+8mMDCQxMREAJYtW0ZISEit57j++uuZOnUqXbvW7WP/iy66iPz8fBYsWFD3wJXvSOgCm+ZCaUnVrfZyN2pJiPJVVWVv5WuBi0gA8AwwudYTidwM3AyQnJzcQOE1DSJCp8QoOiVGcUV6BwDyCopYmXWAlVkHWLZtP28vzaKw+EQXkuiwIDonRlVIuDslRpIcH0FYsI50K3WqNLn2oJYtW7Jqle1T/NBDDxEVFcU991QcnDHGYIwhIKDqSSkzZsyo8+vu27ePn3/+mbCwMLZv3+6xX0YlJSUEBem3kEcldoOyYjiwDRLSKj5XWmInNHa5wDuxKVWzbKCDy+MkIMflcTTQC/jOMYraBpgtImMqT2o0xkwDpgGkp6cbVI0SokK5oGcbLujZBoCyMsOu/EK27C1gS24BW3OPsCW3gB8255aXlACIQLsW4aQmRJDa0k6sTG0ZSWqCTbxDgnTypFLu0MzICzZv3sy4ceMYNmwYS5cu5dNPP+Xhhx/mxx9/5NixY1x55ZU8+OCDAAwbNoznnnuOXr16kZCQwK233sqcOXOIiIjg448/plWrVied//3332fcuHG0aNGCd999l3vvvRewo+e33HIL27ZtQ0SYNm0agwYNYsaMGTzzzDOICP3792fGjBlMmjSJCRMmMG7cOACioqIoKCjg66+/5vHHHychIYG1a9fy888/c8kll5CTk0NhYSF33XUXN954IwCfffYZf/7znyktLaV169bMmTOHrl27smzZMuLj4yktLSUtLY0VK1YQHx/fSO++n0nsYu9zN5ycXB/YZhNv7RSifNNyIE1EOgI7gauAq51PGmMOAQnOxyLyHXBPc+4W4ikBAUL72HDax4YzvEtihecOFxazLe8IW3OPkLnvCJl5R9i27yifrtnFoWPFJ84h0DEhknO7tWJkjzYMSIkjMEBLS5SqSrNJrh/+ZC0ZOfkNes4e7WL4yyU963VsRkYGM2bM4KWXXgLg8ccfJz4+npKSEkaMGMGECRPo0aNHhWMOHTrE2WefzeOPP87dd9/N9OnTmTp16knnnjlzJn//+99p0aIFkyZNKk+ub7/9dkaOHMmUKVMoKSnh6NGjrF69mn/84x8sWrSI+Ph49u/fX2vsS5YsISMjo3xE/I033iA+Pp6jR4+Snp7OZZddRlFREbfddhsLFiwgJSWF/fv3ExgYyMSJE3nnnXeYMmUKX375JWeccYYm1jVJcCbX66H7xRWfc3YK0eRa+SBjTImITAG+xLbim26MWSsijwArjDGzvRuhAogOC6Z3Uiy9k2JPeu7AkeNscyTcmXlHWJ19iDcWZfHKgm20jAzh3G6tOL9nG85KS9ByEqVcNJvk2td07tyZM844o/zxzJkzee211ygpKSEnJ4eMjIyTkuvw8HBGjx4NwIABA6qsp965cyfbt2/nzDPPREQoLS1l/fr1dOvWje+++45Zs2YBEBQURExMDN9++y1XXnlleYLrTqI7ePDgCqUmzzzzDLNn29+T2dnZbNmyhR07djBixAhSUlIqnPc3v/kNl19+OVOmTGH69Onlo9yqGqHREJNUdTs+ZxcRZwKulI8xxnwOfF5p24PV7HtOY8Sk3BcXGUJcZAj9k+PKtx0uLOb7jbnMzdjDF2t3897KbMKCAxielsjIHq05r3tr4iNrn0+kVFPm0eS6tgUERGQy8CT2I0OA54wxrzqeKwV+dmzfbowZcyqx1HeE2VMiIyPLv960aRP//Oc/WbZsGbGxsUyaNKnKhW9cJ0AGBgZSUlJy0j7vvvsu+/btK+/ffOjQIWbNmsVDDz0EnNwlwxhT5azxoKAgysrsBJjS0tIKr+Ua+9dff838+fNZsmQJ4eHhDBs2jMLCwmrPm5qaSlxcHPPmzeOnn37i/PPPr/L9US4Su1Tdji9vI8S0twm4Uko1guiwYC7u3Y6Le7fjeEkZy7bt56uM3czN2MNXGbZLSYf4cLq1iaF72xi6t4mme9sYkuMjCNAyEtVMeCy5dncBAeBdY8yUKk5xzBjT11Px+ZL8/Hyio6OJiYlh165dfPnll4waNape55o5cyZff/11+aj4pk2buPjii3nooYcYMWIEL730ElOmTKG0tJQjR47wq1/9iiuuuII77rijvCwkPj6e1NRUVq5cyaWXXsqHH35IaWlpla936NAh4uPjCQ8PZ+3atSxfvhyAoUOHcuedd5KVlVVeFuI6en3NNddw/fXXVzuRU7lI7AYrX4eyMnB9v3LXa0mIUsprQoICGJaWwLC0BB4e05NfduYzf1Mu63bls373Yb5Zt4cyx/TTiJBAujoS7W5toukQH0H72HDaxYYTpcu5qybGk9/R7i4g0Oz179+fHj160KtXLzp16sTQoUPrdZ4tW7awe/du0tPTy7elpaURGhrKypUree6557jpppt4+eWXCQoK4uWXX2bgwIH84Q9/YPjw4QQFBTFgwABee+01brnlFsaOHcvcuXM5//zzCQ0NrfI1L7roIqZNm0afPn3o1q0bgwbZNratW7fmxRdfZOzYsRhjaNeuHXPmzAFg/Pjx3HDDDUyePLle/85mJ6ELFB+F/GyIdZTjlJVB3iboP8S7sSmlFPZT0dOTWnB6UovybYXFpWzcc5j1uw6TsSufdbvy+XR1Du8srfipa0xYEO0cEy7bld/C6JQQRedWkUSEaPKt/IsY45muRiIyARhljLnR8fjXwCDXUWpHWcjfgVxgI3CXMWaH47kSYBVQAjxujPmoitdw7X06ICsrq8Lz69ato3v37g3/j1OnZMmSJdx3333MmzfPq3H4zfdH1mKYMQqueR/SRtptB7Lgn73h4mch/XrvxqdOIiIrjTHpte+p6io9Pd2sWKENRfyVMYY9+UXsPHiUnQcLyTl4rPzmfOzapQSgfWw4nVtFcVpiFKe1OnHT2m7lTTVd5z3552CNCwg4fALMNMYUicitwBvAuY7nko0xOSLSCfhWRH42xmypcDLtfep3HnvsMaZNm1Y+sVK5wVn6kbvhRHLtnOCoZSFKKT8iIrRpEUabFmEMSKl6n4KiEnIOHmNrbgGb99rbpr0FLNu2r8JiOPGRIbSMDCE8JJCw4EDCnTfXxyEBtI+NoG+HWLq0jiIoUEsRled5MrmubQEBjDH7XB6+AvzD5bkcx/1WR//TfkCF5Fr5n/vvv5/777/f22H4l4h4iEioOKnR2YZPV2dUSjUxUaFBdGkdTZfWFSdrl5UZdh48xubcgvIFcQ4dK+bY8VKOFZdy8Fgxuw8VcqzYPi48XsrR4lJKHYXf4cGBnN6+BX2TY+nbIZY+HWJp1yJMl4JXDc6TyXWNCwgAiEhbY8wux8MxwDrH9jjgqGNEOwEYCjzhwViV8m2J3Sq248tdbxPuyJbei0kppRpRQIDQIT6CDvERjOh68gJqVTHGkLXvKKuzD/LT9oOszj7I6wszOV5qR8ATo0PpkxRL76QWdGkdXT7ZUhfIUafCY8m1mwsI3CEiY7B11fuByY7DuwMvi0gZEICtudaJkKr5SuwCv/wPjLFrFOdt1JIQpZSqhYiQmmCXcB/btz0Ax0vKWLcrn9XZB1m1/SCrdhzk63V7yo8JCw4oHznv1ubEfWJ0qI5yK7d4dApubQsIGGPuA+6r4rhFwOmejE0pv5LYDQoPQsFeiGplR657XebtqJRSyu+EBAXQx1EWcu1gu+1IUQmb9hawcfdh1u8+zMY9h/luQy7vr8wuPy46NIjWLcJoExNGq5hQ2sTY2vHWMXZb65gwWkaFEBQgmoQ3c9rfRil/4FyFMc9Ra114SOutlVKqgUSGBtG3g63FdrWvoIgNew6zcfdhtuUdYU9+EbvzC9m6pYC9h4soKau6l4IIBIoQIEJAAASI2McBQkhQAO1jw0ltGUFyy0hS4iNITYggOT6ShKgQTcybAE2uPeicc87hvvvu44ILLijf9uyzz7Jx40ZeeOGFao+LioqioKCAnJwc7rjjDt5///0qz/3UU09V6Gld2bPPPsvNN99MREQEABdeeCHvvPMOsbGx1R5TF3369KFHjx7MnDmzQc6nauDaMaR8my57rpRSntQyKpQhUaEM6Zxw0nNlZYa8I0XszS9i96FCducXcuDIcUqNoazM2Htj9yt1PDYGjh0vZceBoyzPPMDs1Tm45ueRIYF0iI+gU2IkI3u05oKebbTPtx/S/zEPmjhxIrNmzaqQXM+aNYsnn3zSrePbtWtXZWLtrmeffZZJkyaVJ9eff/55LUe4b926dZSVlTF//nyOHDlSYUn0hlRSUkJQkH6bEt0WQmMqJdfdvBePUko1cwEBQqvoMFpFh9GrfYvaD6jC8ZIysg8cJWvfUbL2HSFr/1G27zvKqu0H+fzn3USG/MKFp7flsgFJDEyNr/MS8qVlhgBBR8MbmWYtHjRhwgQeeOABioqKCA0NJTMzk5ycHIYNG0ZBQQFjx47lwIEDFBcX8+ijjzJ27NgKx2dmZnLxxRfzyy+/cOzYMa6//noyMjLo3r07x44dK9/vtttuY/ny5Rw7dowJEybw8MMP869//YucnBxGjBhBQkIC8+bNIzU1lRUrVpCQkMDTTz/N9OnTAbjxxhu58847yczMZPTo0QwbNoxFixbRvn17Pv74Y8LDw0/6t73zzjv8+te/Zt26dcyePZuJEycCsHnzZm699VZyc3MJDAzkvffeo3PnzjzxxBO89dZbBAQEMHr0aB5//PEKo+95eXmkp6eTmZnJ66+/zmeffUZhYSFHjhxh9uzZ1b5Xb775Jk899RQiQu/evXnhhRfo3bs3GzduJDg4mPz8fHr37s2mTZsIDg721H+154nY0hBnWUhItE24lVJK+a2QoAA6JUbRKTGqwvayMsPyzP188GM2n/+8m/dWZpMUF86l/ZO4rH97UlqePKB17Hgp63bnszYnn4ycQ6zNscvQC1SoDa/4dSitY8Jo2yJcO6Q0oOaTXM+ZCrt/bthztjkdRj9e7dMtW7Zk4MCBfPHFF4wdO5ZZs2Zx5ZVXIiKEhYXx4YcfEhMTQ15eHmeeeSZjxoyp9q/LF198kYiICNasWcOaNWvo379/+XOPPfYY8fHxlJaWct5557FmzRruuOMOnn76aebNm0dCQsWPs1auXMmMGTNYunQpxhgGDRrE2WefTVxcHJs2bWLmzJm88sorXHHFFXzwwQdMmjTppHjeffdd5s6dy4YNG3juuefKk+trrrmGqVOnMn78eAoLCykrK2POnDl89NFHLF26lIiICPbv31/rW7t48WLWrFlDfHw8JSUlVb5XGRkZPPbYYyxcuJCEhAT2799PdHQ055xzDp999hnjxo1j1qxZXHbZZf6dWDsldoPNc23HkMSuNuFWSinV5AQECIM6tWRQp5Y8PKYXX67dzQc/ZvPvbzfxr282cUZqHGP6tqeouJRfdtpEektuQXmJSWxEMD3bxXDdYLtSz+78IvYcKmTVjoPsXlvI8ZKyCq8XGRJI76RY+ibH0q+DvW8VHdbY/+wmo/kk117iLA1xJtfO0WJjDH/605+YP38+AQEB7Ny5kz179tCmTZsqzzN//nzuuOMOAHr37k3v3r3Ln/vvf//LtGnTKCkpYdeuXWRkZFR4vrIffviB8ePHl5dyXHrppSxYsIAxY8bQsWNH+vbtC8CAAQPIzMw86fjly5eTmJhISkoKSUlJ3HDDDRw4cICgoCB27tzJ+PHjAQgLsz+YX3/9Nddff315eUp8fHyt79vIkSPL96vuvfr222+ZMGFC+R8Pzv1vvPFGnnjiCcaNG8eMGTN45ZVXan09v5DYBVb9B4qPQfdLvB2NUkqpRhAeEsi4fu0Z1689OQeP8eFPO/ngx2z+/NEvALSJCaNnuxhGn96Wnu1i6NW+RY2L4xhjOHi0mN35tk5896FC1u/K56cdB3ll/tbySZrtY8Pp51hwp19yHKclRhEVFqQj3G5oPsl1DSPMnjRu3DjuvvtufvzxR44dO1Y+4vz222+Tm5vLypUrCQ4OJjU1lcLCwhrPVdUPyrZt23jqqadYvnw5cXFxTJ48udbzGFP9SvGhoaHlXwcGBlYoP3GaOXMm69evJzU1FYD8/Hw++OADrrjiimpfr6rYg4KCKCuzfz1Xjtm1hru696q68w4dOpTMzEy+//57SktL6dWrV7X/Xr/irLEuytce10op1Qy1iw3n9hGn8X/ndGZLbgGxESEkRIXWfqALESEuMoS4yBC6t42p8FxhcSlrcw7x0/aD5bdP1+yqsE9kSCBRYUFEhQYRFRZMdKjz6yAiQwIJCQqwt0DXr6X868iQIJLiIugQH050WBP4VLkKzSe59pKoqCjOOeccbrjhhvLSCYBDhw7RqlUrgoODmTdvHllZWTWeZ/jw4bz99tuMGDGCX375hTVr1gA2sY2MjKRFixbs2bOHOXPmcM455wAQHR3N4cOHTyoLGT58OJMnT2bq1KkYY/jwww9566233Pr3lJWV8d5777FmzRrat7cN+efNm8ejjz7KjTfeSFJSEh999BHjxo2jqKiI0tJSzj//fB555BGuvvrq8rKQ+Ph4UlNTWblyJQMHDqxx4mZ179V5553H+PHjueuuu2jZsmX5eQGuvfZaJk6cyJ///Ge3/l1+IcGlO4i24VNKqWZLRDitVXTtO9ZRWHAgA1LiGZBy4hPmPfmF/LT9INkHjnK4sISCohIKHPeHi0ooKCxm7+FCCgpLOHK8lOMlZRwvLStfdr4mcRHB5atudnAk3MnxEbSLDScsOJDgQCEkMICgwACCA4XggIA6T+r0Bk2uG8HEiRO59NJLmTVrVvm2a665hksuuYT09HT69u1Lt241d3647bbbuP766+nduzd9+/Zl4MCBgG2H169fP3r27EmnTp0YOnRo+TE333wzo0ePpm3btsybN698e//+/Zk8eXL5OW688Ub69etXZQlIZfPnz6d9+/bliTXYZD0jI4Ndu3bx1ltvccstt/Dggw8SHBzMe++9x6hRo1i1ahXp6emEhIRw4YUX8re//Y177rmHK664grfeeotzzz232tes7r3q2bMn999/P2effTaBgYH069eP119/vfyYBx54oMIfNH4vNhmCwqCkUNvwKaWUahStY8IY1avqktWalJYZm2iXlFFUWlr+dUFRCdkHjrF9/1F27D/K9v1HycjJ56u1uykurT0hDwwQggOF0KBAUltG0LlVFGmtoklrFUVa6yiS4ry/fL3UVCLgT9LT082KFSsqbFu3bh3du3f3UkTKm95//30+/vjjGkfk/fL746VhkLcJ/pQDAYHejkZVQ0RWGmOqb0Kv6q2qa71Syv+Vlhn25Beyff9Rdh8qLB8BLy4to6TUlH9tb4ajx0vIzDvKpr2H2ZNfVH6eUEcHlrRWUXROjEIEO8pePupeXPFxUQlXnZHM1NF1a29b03VeR65Vk/Pb3/6WOXPmNGhfb5/R8WyIaqOJtVJKqSYlMEBoFxtOu9iT2//WJr+wmM17C9i8p4BNew+zaW8BK7PsIj0AYcEBRIUGExMWVF4vnhwfQVRYENGO1TkbkibXqsn597//7e0QPOeCx7wdgVJKKeVTYsKC6Z8cR//kuArbC4tLHWUkAY0ajybXSimllFKqyQkL9s6nvI2byntBU6kpVw1Lvy+UUkop5QlNOrkOCwtj3759mkipCowx7Nu3r3yRG6WUUkqphtKky0KSkpLIzs4mNzfX26EoHxMWFkZSUpK3w1CqSRORUcA/gUDgVWPM45Wevxu4ESgBcoEbjDE1N/1XSikf59Hk2o0L62TgSWCnY9NzxphXHc9dBzzg2P6oMeaNur5+cHAwHTt2rGf0Siml6ktEAoHngZFANrBcRGYbYzJcdvsJSDfGHBWR24AngCsbP1qllGo4Hkuu3bywArxrjJlS6dh44C9AOmCAlY5jD3gqXqWUUg1qILDZGLMVQERmAWOB8t8Bxph5LvsvASY1aoRKKeUBnqy5Lr+wGmOOA84LqzsuAOYaY/Y7Euq5wCgPxamUUqrhtQd2uDzOdmyrzm+AOVU9ISI3i8gKEVmhZX5KKV/nyeTa3QvrZSKyRkTeF5EOdTxWKaWUb6pq/eEqZ5eLyCTsJ5VPVvW8MWaaMSbdGJOemJjYgCEqpVTD82TNtTsX1k+AmcaYIhG5FXgDONfNYxGRm4GbHQ8LRGRDpV0SgLw6Re19/hgz+Gfc/hgzaNyNqb4xpzR0IH4oG+jg8jgJyKm8k4j8CrgfONsYU1T5+cpWrlyZJyKukx798fsK/DNuf4wZNO7G5I8xQ/3irvY678nkutYLqzFmn8vDV4B/uBx7TqVjv6v8AsaYacC06gIQkRXVrfvuq/wxZvDPuP0xZtC4G5M/xuxDlgNpItIRO2n9KuBq1x1EpB/wMjDKGLPXnZMaYyoMXfvr/5E/xu2PMYPG3Zj8MWZo+Lg9WRZSfmEVkRDshXW26w4i0tbl4RhgnePrL4HzRSROROKA8x3blFJK+QFjTAkwBXvtXgf81xizVkQeEZExjt2eBKKA90RklYjMruZ0SinlNzw2cm2MKRER54U1EJjuvLACK4wxs4E7HBfZEmA/MNlx7H4R+Ss2QQd4xBiz31OxKqWUanjGmM+Bzytte9Dl6181elBKKeVhHu1z7caF9T7gvmqOnQ5MP8UQqi0Z8WH+GDP4Z9z+GDNo3I3JH2Nubvz1/8gf4/bHmEHjbkz+GDM0cNyiS4MrpZRSSinVMDxZc62UUkoppVSz0iSTaxEZJSIbRGSziEz1djzuEpFMEfnZMbFnhbfjqY6ITBeRvSLyi8u2eBGZKyKbHPdx3oyxsmpifkhEdjre71UicqE3Y6yKiHQQkXkisk5E1orI7xzbffb9riFmn36/RSRMRJaJyGpH3A87tncUkaWO9/pdxwRt5QP88Vqv13nP8sdrvT9e58E/r/WNdZ1vcmUhYpdd34jLsuvAxCqWXfc5IpIJpBtjfLpHpIgMBwqAN40xvRzbngD2G2Med/ySizPG/NGbcbqqJuaHgAJjzFPejK0mjo46bY0xP4pINLASGIed/OuT73cNMV+BD7/fIiJApDGmQESCgR+A3wF3A/8zxswSkZeA1caYF70Zq/Lfa71e5z3LH6/1/nidB/+81jfWdb4pjlyfyrLryg3GmPnY7i6uxmIXAcJxP65Rg6pFNTH7PGPMLmPMj46vD2NbmrXHh9/vGmL2acYqcDwMdtwMdmGr9x3bfeq9bub0Wu9B/nidB/+81vvjdR7881rfWNf5pphc+/PS6Qb4SkRWil190p+0NsbsAvsDB7TycjzumiIiaxwfJfrUR26ViUgq0A9Yip+835ViBh9/v0UkUERWAXuBucAW4KCjZzP41/WkqfPXa71e573Dp689Tv54nQf/utY3xnW+KSbXbi2d7qOGGmP6A6OB2x0fbynPeRHoDPQFdgH/z7vhVE9EooAPgDuNMfnejscdVcTs8++3MabUGNMXuyrsQKB7Vbs1blSqGv56rdfrfOPz+WsP+Od1HvzvWt8Y1/mmmFzXuuy6rzLG5Dju9wIfYv/T/cUeR/2Vsw7LraWMvckYs8fxQ1YGvIKPvt+OurAPgLeNMf9zbPbp97uqmP3l/QYwxhwEvgPOBGJFxLkmgN9cT5oBv7zW63W+8fnDtccfr/Pg39d6T17nm2JyXeuy675IRCIdEwIQkUjsku+/1HyUT5kNXOf4+jrgYy/G4hbnRcthPD74fjsmX7wGrDPGPO3ylM++39XF7Ovvt4gkikis4+tw4FfYGsJ5wATHbj71Xjdzfnet1+u8d/jBtcfvrvPgn9f6xrrON7luIQCOti/PcmLZ9ce8HFKtRKQTdhQD7MqZ7/hq3HHXifsAAAK7SURBVCIyEzgHSAD2AH8BPgL+CyQD24HLfWnJ+mpiPgf7sZUBMoFbnPVtvkJEhgELgJ+BMsfmP2Hr2nzy/a4h5on48PstIr2xE1kCsQMP/zXGPOL42ZwFxAM/AZOMMUXei1Q5+du1Xq/znueP13p/vM6Df17rG+s63ySTa6WUUkoppbyhKZaFKKWUUkop5RWaXCullFJKKdVANLlWSimllFKqgWhyrZRSSimlVAPR5FoppZRSSqkGosm1apJEpFREVrncpjbguVNFxGf6diqlVHOk13nlq4Jq30Upv3TMsbypUkqppkmv88on6ci1alZEJFNE/iEiyxy30xzbU0TkGxFZ47hPdmxvLSIfishqx+3/t3fHqlFEURzGvz8hSEBstBEsbFIJChIsLH0FiyhWkipNrMQX8AmCNhEsBN9BFAtBFDubtGKXQFKIpBGRY7FXGCVBkLvOsvv9mjlzZlnuNIczd+/svd6+ainJkyS7SV62nZ4kSSOzzmtsNteaVyt//Fy4Prj2taquAY+Y7O5Gi59V1WXgObDd8tvAm6q6AlwFdlt+FXhcVZeAL8DNKd+PJOl31nnNJHdo1FxKclRVp4/JfwZuVNWnJMvAflWdTXIInK+q7y2/V1XnkhwAF4bboCa5CLyqqtV2/gBYrqqH078zSRJY5zW7nLnWIqoT4pM+c5xvg/gHvr8gSbPEOq/R2FxrEa0Pju9b/A641eI7wNsWvwY2AZIsJTnzvwYpSfpn1nmNxqcwzauVJB8H5y+q6tffNJ1K8oHJw+XtltsCnia5DxwAd1v+HrCTZIPJzMUmsDf10UuS/sY6r5nkmmstlLYWb62qDsceiySpP+u8xuayEEmSJKkTZ64lSZKkTpy5liRJkjqxuZYkSZI6sbmWJEmSOrG5liRJkjqxuZYkSZI6sbmWJEmSOvkJPPzCRFd0v9oAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#可視化結果\n", + "f, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n", + "t = f.suptitle('Basic CNN Performance', fontsize=12)\n", + "f.subplots_adjust(top=0.85, wspace=0.3)\n", + "\n", + "epoch_list = list(range(1,31))\n", + "ax1.plot(epoch_list, history.history['acc'], label='Train Accuracy')\n", + "ax1.plot(epoch_list, history.history['val_acc'], label='Validation Accuracy')\n", + "ax1.set_xticks(np.arange(0, 31, 5))\n", + "ax1.set_ylabel('Accuracy Value')\n", + "ax1.set_xlabel('Epoch')\n", + "ax1.set_title('Accuracy')\n", + "l1 = ax1.legend(loc=\"best\")\n", + "\n", + "ax2.plot(epoch_list, history.history['loss'], label='Train Loss')\n", + "ax2.plot(epoch_list, history.history['val_loss'], label='Validation Loss')\n", + "ax2.set_xticks(np.arange(0, 31, 5))\n", + "ax2.set_ylabel('Loss Value')\n", + "ax2.set_xlabel('Epoch')\n", + "ax2.set_title('Loss')\n", + "l2 = ax2.legend(loc=\"best\")" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Model: \"sequential_2\"\n", + "_________________________________________________________________\n", + "Layer (type) Output Shape Param # \n", + "=================================================================\n", + "conv2d_5 (Conv2D) (None, 98, 98, 32) 896 \n", + "_________________________________________________________________\n", + "max_pooling2d_5 (MaxPooling2 (None, 49, 49, 32) 0 \n", + "_________________________________________________________________\n", + "conv2d_6 (Conv2D) (None, 47, 47, 64) 18496 \n", + "_________________________________________________________________\n", + "max_pooling2d_6 (MaxPooling2 (None, 23, 23, 64) 0 \n", + "_________________________________________________________________\n", + "conv2d_7 (Conv2D) (None, 21, 21, 128) 73856 \n", + "_________________________________________________________________\n", + "max_pooling2d_7 (MaxPooling2 (None, 10, 10, 128) 0 \n", + "_________________________________________________________________\n", + "conv2d_8 (Conv2D) (None, 8, 8, 128) 147584 \n", + "_________________________________________________________________\n", + "max_pooling2d_8 (MaxPooling2 (None, 4, 4, 128) 0 \n", + "_________________________________________________________________\n", + "flatten_2 (Flatten) (None, 2048) 0 \n", + "_________________________________________________________________\n", + "dropout_1 (Dropout) (None, 2048) 0 \n", + "_________________________________________________________________\n", + "dense_3 (Dense) (None, 512) 1049088 \n", + "_________________________________________________________________\n", + "dense_4 (Dense) (None, 3) 1539 \n", + "=================================================================\n", + "Total params: 1,291,459\n", + "Trainable params: 1,291,459\n", + "Non-trainable params: 0\n", + "_________________________________________________________________\n" + ] + } + ], + "source": [ + "#套入VGG16模型\n", + "from keras.applications import VGG16\n", + "\n", + "\n", + "conv_base = VGG16(weights='imagenet',\n", + " include_top=False,\n", + " input_shape=(100,100, 3))\n", + "model.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Found 60 images belonging to 3 classes.\n", + "[[1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [1. 0. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]]\n", + "[[0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 1. 0.]]\n", + "[[0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 1. 0.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [0. 0. 1.]]\n", + "Found 27 images belonging to 3 classes.\n", + "[[0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [1. 0. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]]\n", + "Found 15 images belonging to 3 classes.\n", + "[[0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [1. 0. 0.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]\n", + " [0. 0. 1.]\n", + " [0. 0. 1.]\n", + " [0. 1. 0.]\n", + " [1. 0. 0.]]\n" + ] + } + ], + "source": [ + "#使用conv_base的模型先抽取特徵跟label產出，將特徵提取後進行降維輸出\n", + "import os\n", + "import numpy as np\n", + "from keras.preprocessing.image import ImageDataGenerator\n", + "\n", + "base_dir = 'Rec'\n", + "\n", + "train_dir = os.path.join(base_dir, 'train')\n", + "validation_dir = os.path.join(base_dir, 'validation')\n", + "test_dir = os.path.join(base_dir, 'test')\n", + "\n", + "datagen = ImageDataGenerator(rescale=1./255)\n", + "batch_size = 20\n", + "\n", + "def extract_features(directory, sample_count):\n", + " features = np.zeros(shape=(sample_count, 3, 3, 512))\n", + " labels = np.zeros(shape=(sample_count,3))\n", + " generator = datagen.flow_from_directory(\n", + " directory,\n", + " target_size=(100, 100),\n", + " batch_size=batch_size,\n", + " class_mode='categorical'\n", + " )\n", + " i = 0\n", + " for inputs_batch, labels_batch in generator:\n", + " features_batch = conv_base.predict(inputs_batch)\n", + " features[i * batch_size : (i + 1) * batch_size] = features_batch\n", + " print(labels_batch)\n", + " labels[i * batch_size : (i + 1) * batch_size] = labels_batch\n", + " i += 1\n", + " if i * batch_size >= sample_count:\n", + " \n", + " break\n", + " return features, labels\n", + "\n", + "train_features, train_labels = extract_features(train_dir, 60)\n", + "validation_features, validation_labels = extract_features(validation_dir, 20)\n", + "test_features, test_labels = extract_features(test_dir, 15)" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "train_features = np.resize(train_features, (60, 3 * 3 * 512))\n", + "validation_features = np.resize(validation_features, (20, 3 * 3 * 512))\n", + "test_features = np.resize(test_features, (15, 3 * 3 * 512))" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 60 samples, validate on 20 samples\n", + "Epoch 1/30\n", + "60/60 [==============================] - 1s 10ms/step - loss: 2.0313 - acc: 0.3167 - val_loss: 1.8868 - val_acc: 0.3000\n", + "Epoch 2/30\n", + "60/60 [==============================] - 0s 483us/step - loss: 1.8181 - acc: 0.3333 - val_loss: 1.7381 - val_acc: 0.3000\n", + "Epoch 3/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 1.6853 - acc: 0.3000 - val_loss: 1.6217 - val_acc: 0.3000\n", + "Epoch 4/30\n", + "60/60 [==============================] - 0s 550us/step - loss: 1.6355 - acc: 0.3167 - val_loss: 1.5229 - val_acc: 0.3000\n", + "Epoch 5/30\n", + "60/60 [==============================] - 0s 483us/step - loss: 1.5393 - acc: 0.3333 - val_loss: 1.4473 - val_acc: 0.3000\n", + "Epoch 6/30\n", + "60/60 [==============================] - 0s 550us/step - loss: 1.5782 - acc: 0.2833 - val_loss: 1.3734 - val_acc: 0.3000\n", + "Epoch 7/30\n", + "60/60 [==============================] - 0s 533us/step - loss: 1.4986 - acc: 0.2833 - val_loss: 1.3062 - val_acc: 0.3000\n", + "Epoch 8/30\n", + "60/60 [==============================] - 0s 500us/step - loss: 1.4321 - acc: 0.3667 - val_loss: 1.2484 - val_acc: 0.3000\n", + "Epoch 9/30\n", + "60/60 [==============================] - 0s 583us/step - loss: 1.3253 - acc: 0.3333 - val_loss: 1.1981 - val_acc: 0.3000\n", + "Epoch 10/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 1.2468 - acc: 0.4167 - val_loss: 1.1543 - val_acc: 0.3000\n", + "Epoch 11/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 1.1066 - acc: 0.4500 - val_loss: 1.1124 - val_acc: 0.3000\n", + "Epoch 12/30\n", + "60/60 [==============================] - 0s 600us/step - loss: 1.2571 - acc: 0.3333 - val_loss: 1.0764 - val_acc: 0.3000\n", + "Epoch 13/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 1.1691 - acc: 0.4667 - val_loss: 1.0375 - val_acc: 0.3500\n", + "Epoch 14/30\n", + "60/60 [==============================] - 0s 550us/step - loss: 1.1005 - acc: 0.5167 - val_loss: 1.0097 - val_acc: 0.3500\n", + "Epoch 15/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 1.1293 - acc: 0.4167 - val_loss: 0.9818 - val_acc: 0.4000\n", + "Epoch 16/30\n", + "60/60 [==============================] - 0s 516us/step - loss: 1.1421 - acc: 0.4167 - val_loss: 0.9571 - val_acc: 0.4000\n", + "Epoch 17/30\n", + "60/60 [==============================] - 0s 533us/step - loss: 1.0784 - acc: 0.5000 - val_loss: 0.9312 - val_acc: 0.5500\n", + "Epoch 18/30\n", + "60/60 [==============================] - 0s 516us/step - loss: 1.0378 - acc: 0.4500 - val_loss: 0.9119 - val_acc: 0.6000\n", + "Epoch 19/30\n", + "60/60 [==============================] - 0s 533us/step - loss: 0.9446 - acc: 0.5000 - val_loss: 0.8883 - val_acc: 0.7500\n", + "Epoch 20/30\n", + "60/60 [==============================] - 0s 533us/step - loss: 0.9756 - acc: 0.5167 - val_loss: 0.8679 - val_acc: 0.7500\n", + "Epoch 21/30\n", + "60/60 [==============================] - 0s 633us/step - loss: 0.9451 - acc: 0.5667 - val_loss: 0.8540 - val_acc: 0.7500\n", + "Epoch 22/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 0.7960 - acc: 0.6500 - val_loss: 0.8393 - val_acc: 0.7500\n", + "Epoch 23/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 0.8670 - acc: 0.5833 - val_loss: 0.8223 - val_acc: 0.7500\n", + "Epoch 24/30\n", + "60/60 [==============================] - 0s 600us/step - loss: 0.9493 - acc: 0.6167 - val_loss: 0.8102 - val_acc: 0.7500\n", + "Epoch 25/30\n", + "60/60 [==============================] - 0s 566us/step - loss: 0.9749 - acc: 0.5500 - val_loss: 0.7922 - val_acc: 0.7500\n", + "Epoch 26/30\n", + "60/60 [==============================] - 0s 583us/step - loss: 0.9339 - acc: 0.6333 - val_loss: 0.7809 - val_acc: 0.8000\n", + "Epoch 27/30\n", + "60/60 [==============================] - 0s 550us/step - loss: 0.7419 - acc: 0.7000 - val_loss: 0.7721 - val_acc: 0.8000\n", + "Epoch 28/30\n", + "60/60 [==============================] - 0s 583us/step - loss: 0.8690 - acc: 0.6167 - val_loss: 0.7634 - val_acc: 0.8000\n", + "Epoch 29/30\n", + "60/60 [==============================] - 0s 583us/step - loss: 0.8348 - acc: 0.6167 - val_loss: 0.7556 - val_acc: 0.8000\n", + "Epoch 30/30\n", + "60/60 [==============================] - 0s 533us/step - loss: 0.7058 - acc: 0.7000 - val_loss: 0.7460 - val_acc: 0.8000\n" + ] + } + ], + "source": [ + "#建立優化防止梯度下降訓練\n", + "from keras import models\n", + "from keras import layers\n", + "from keras import optimizers\n", + "\n", + "model = models.Sequential()\n", + "model.add(layers.Dense(256, activation='relu', input_dim=3 * 3 * 512))\n", + "model.add(layers.Dropout(0.5))\n", + "model.add(layers.Dense(3, activation='softmax'))\n", + "\n", + "model.compile(optimizer=optimizers.RMSprop(lr=2e-5),\n", + " loss='categorical_crossentropy',\n", + " metrics=['acc'])\n", + "\n", + "history = model.fit(train_features, train_labels,\n", + " epochs=30,\n", + " batch_size=100,\n", + " validation_data=(validation_features, validation_labels))" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "model.save('train3.h5')" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtAAAAEjCAYAAAAbuTa2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3iUVfbA8e9JIwQSIA2EAEFqACkhUgTpsiIqIqyCYEGxrW2Lrui6rrr6s3dcFRVsCBYUcRVRuopSQhNIqAYJLSEBAoEASc7vj3eSDZAygZlMEs7neeYh88597z0zCTNn7nuLqCrGGGOMMcYY9/j5OgBjjDHGGGOqEkugjTHGGGOMKQdLoI0xxhhjjCkHS6CNMcYYY4wpB0ugjTHGGGOMKQdLoI0xxhhjjCkHS6CNMWctEZklItf7Og5fEMdkEdknIkt9HY8xxlQllkAbYyo9EUkRkSMicsiV8H0tIo3PtF5VHayq751GPCIid4vIWhHJFpFUEflURM5zPf6uiKiIdC1yTgsR0SL3F4hITtHnISIDRSSllHbV1d4hEdkhIi+IiH9543fpBVwExKhq17IKG2OM+R9LoI0xVcVlqlobOAfYA7zqw1heBu4B7gbCgVbADGBIkTKZwONl1JMN/LOcbXd0vQ4DgGuAm8t5PiISADQFUlQ1+zTPN8aYs5Yl0MaYKkVVc4DPgLYFx0RkiIisFJEsEdkuIo8UeSxYRD4UkQwR2S8iy0SkvuuxBSIyrkjZm0UkSUQOish6EYk/uX0RaQncAYxS1XmqelRVD6vqFFV9qkjR94AOItKnlKfzCjBKRFqcxuuQDPwAtHfF1VBEpotIuoj8JiJ3F4n5ERH5zPU6ZAE3AW8DPVy92Y8Wef6bRSRTRGaKSMMidaiI3CEim4BNRY79SUQ2uV6zf4tIcxH52fW7+EREglxl64nIf13x7XP9HFOk/gWu839y1fWdiEQWebyXiCx2/Q63i8gNruM1ROQ5EfldRPaIyBsiUrO8r6cxxpSHJdDGmCpFREKAq4FfihzOBq4D6uL0At8uIle4HrseqAM0BiKA24AjxdT7R+ARVz1hwOVARjEhDABSVbWsccOHgf8DniilzA7gLVe75SIibYELgZUi4gd8BawGGrli/LOI/KHIKUNxvnjUBd7HeR1+VtXaqvovEekPPAlchdPLvw2YdlKzVwDdKPLlBbgY6AJ0B/4OTARG47ze7YFRrnJ+wGScnu8mOL+DCSfVfw0wFogGgoB7Xc+1CTAL56pDFNAJWOU652mcKwCdgBau5/9waa+dMcacKUugjTFVxQwR2Q9k4YzdfbbgAVVdoKq/qmq+qq4BpgIFPb/HcRLnFqqap6qJqppVTP3jgGdUdZk6NqvqtmLKRQC73Iz5TaCJiAwupcyTwGUi0s7NOleIyD6chPltnKT0fCBKVR9T1WOquhUnMR9Z5LyfVXWG6zU65QsETtI7SVVXqOpR4AGcHurYorGqauZJ5z+tqlmqug5YC3ynqltV9QBO0tsZQFUzVHW6q7f+IM4Xi5N75yer6kZX/Z/gJMUFsc1R1amqetxV1yoREZwhLH9xxXUQ50vLSIwxxotsHJsxpqq4QlXnuCbNDQUWikhbVd0tIt2Ap3B6PIOAGsCnrvM+wOkNnSYidYEPgX+o6vGT6m8MbHEjjgycHtoyqepREfk38G/+1xN7cpl0EZkAPAa87ka18aq6uegBEWkKNHR9wSjgjzPEo8D2MuptCKwoEtchEcnA6dFNKaWOPUV+PlLM/QauGEOAF3F6rOu5Hg8VEX9VzXPd313k3MNAbdfPJf1uooAQINHJpQEQnOdujDFeYz3QxpgqxdWL/DmQh7OSBMBHwEygsarWAd7ASaRw9Vg+qqptgQuAS3GGaZxsO9DcjRDmAjEikuBmyJNxhpAMK6XMs0A/nKEQp2M78Juq1i1yC1XVS4qU0ZJOdtmJM7wCABGphdPbvqMcdZTmb0BroJuqhgG9C5py49ySfjd7cZL0dkWedx3XJEtjjPEaS6CNMVWKOIbi9GImuQ6HApmqmuNaOu6aIuX7ich5rp7rLJwhHXkn14szHOJeEeniaqOFq2f3BKq6CfgPMFVE+opIkGui4kgRGV9M+VycMc73l/ScVHU/8DzOGOLTsRTIEpH7RaSmiPiLSHsROb8cdXwEjBWRTiJSA2coxBJVTTnNmE4WipPs7heRcOBf5Th3CjBQRK4SkQARiRCRTqqajzNU5UURiQYQkUYnjf02xhiPswTaGFNVfCUih3CS4CeA613jbgH+BDwmIgdxJpB9UuS8BjiT57JwEu6FOMM4TqCqn7rq/Qg4iLMsXXgJsdyNMwHuNWA/zvCCYTjjkoszlbLHTb9M8Yl9mVxDIC7DGTP8G07P7Ns4Pd/u1jEXZ0m96a5Ym+PZscQvATVdsf0CfFuO2H4HLsHpxc7EmUDY0fXw/cBm4BfXCiNzcHq6jTHGa0T1TK7IGWOMMcYYc3axHmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhjjDHGGGPKwRJoY4wxxhhjysESaGOMMcYYY8rBEmhT6YjIAhHZJyI1fB2LMcaYykFEUkRkoK/jMAYsgTaVjIjEAhcCClxege0GVFRbxhhjjKnaLIE2lc11wC/Au8D1BQdFpKaIPC8i20TkgIj8KCI1XY/1EpHFIrJfRLaLyA2u4wtEZFyROm4QkR+L3FcRuUNENgGbXMdedtWRJSKJInJhkfL+IvKgiGwRkYOuxxuLyGsi8nzRJyEiX4nIn73xAhljjPkfEblZRDaLSKaIzBSRhq7jIiIvikia63NjjYi0dz12iYisd72X7xCRe337LExVYwm0qWyuA6a4bn8Qkfqu488BXYALgHDg70C+iDQBZgGvAlFAJ2BVOdq7AugGtHXdX+aqIxz4CPhURIJdj/0VGAVcAoQBNwKHgfeAUSLiByAikcAAYGp5nrgxxpjyEZH+wJPAVcA5wDZgmuvhQUBvoBVQF7gayHA99g5wq6qGAu2BeRUYtqkGLIE2lYaI9AKaAp+oaiKwBbjGlZjeCNyjqjtUNU9VF6vqUWA0MEdVp6rqcVXNUNXyJNBPqmqmqh4BUNUPXXXkqurzQA2gtavsOOAhVd2gjtWuskuBAzhJM8BIYIGq7jnDl8QYY0zpRgOTVHWF6zPhAaCHazjgcSAUaAOIqiap6i7XeceBtiISpqr7VHWFD2I3VZgl0KYyuR74TlX3uu5/5DoWCQTjJNQna1zCcXdtL3pHRP4mIkmuy337gTqu9stq6z1gjOvnMcAHZxCTMcYY9zTE6XUGQFUP4fQyN1LVecAE4DVgj4hMFJEwV9HhOFcTt4nIQhHpUcFxmyrOEmhTKbjGM18F9BGR3SKyG/gL0BHnslwO0LyYU7eXcBwgGwgpcr9BMWW0SAwXAve74qinqnVxepbFjbY+BIaKSEcgDphRQjljjDGesxPnyiUAIlILiAB2AKjqK6raBWiHM5TjPtfxZao6FIjGeb/+pILjNlWcJdCmsrgCyMMZi9zJdYsDfsAZFz0JeEFEGrom8/VwLXM3BRgoIleJSICIRIhIJ1edq4ArRSRERFoAN5URQyiQC6QDASLyMM5Y5wJvA/8WkZauySkdRCQCQFVTccZPfwBMLxgSYowxxqMCRSS44IaT+I4VkU6uz4T/A5aoaoqInC8i3UQkEKdDJQfIE5EgERktInVU9TiQhfP5Y4zbLIE2lcX1wGRV/V1VdxfccC6/jQbGA7/iJKmZwNOAn6r+jnMZ7m+u46tweq0BXgSOAXtwhlhMKSOG2TgTEjfiXBLM4cQhHi/gvFl/h/OG+w5Qs8jj7wHnYcM3jDHGW74BjhS5XQj8E5gO7MK5SjjSVTYMeAvYh/OenoEzIR3gWiBFRLKA2/jfEDxj3CKqWnYpY0yZRKQ3zlCOWFXN93U8xhhjjPEO64E2xgNclwjvAd625NkYY4yp3iyBNuYMiUgcsB9nsuNLPg7HGGOMMV5mQziMMcYYY4wpB+uBNsYYY4wxphwCfB1AeUVGRmpsbKyvwzDGnOUSExP3qmqUr+Ooruy93hhTGZT0Xl/lEujY2FiWL1/u6zCMMWc5EdlWdilzuuy93hhTGZT0Xm9DOIwxxhhjjCkHS6CNMcYYY4wpB0ugjTHGGGOMKYcqNwa6OMePHyc1NZWcnBxfh2IqkeDgYGJiYggMDPR1KMYYY84Clo9UXeXNGbyaQIvIxcDLgD/ODm1PnfR4E+A9oK6rzHhV/aa87aSmphIaGkpsbCwi4oHITVWnqmRkZJCamkqzZs18HY4xxpizgOUjVdPp5AxeG8IhIv7Aa8BgoC0wSkTanlTsIeATVe0MjAT+czpt5eTkEBERYX+sppCIEBERYb0AxhhjKozlI1XT6eQM3uyB7gpsVtWtACIyDRgKrC9SRoEw1891gJ2n25j9sZqT2d9EJaQK25dCbiX7YlOzHpzTwddRmNOUn6+s2XGAOjUDaRZZy9fhmLOcffZUTeX9vXkzgW4EbC9yPxXodlKZR4DvROQuoBYwsLiKROQW4BaAJk2aeDxQY0wFWT0VZtzu6yhO1bw/XPuFr6MwpyknN4+r3vyZ0d2a8K/L2vk6HGPMWcCbCXRxqbyedH8U8K6qPi8iPYAPRKS9quafcJLqRGAiQEJCwsl1+FxGRgYDBgwAYPfu3fj7+xMV5Wxas3TpUoKCgsqsY+zYsYwfP57WrVuXq+0hQ4aQlZXFDz/8UP7AjalIebmw8BlocB5c/LSvozlRzbq+jsCcgZCgAHo2j2BuUhoPX9rWegDNWcsX+cjbb7/N2rVreemll04/8CrImwl0KtC4yP0YTh2icRNwMYCq/iwiwUAkkObFuDwuIiKCVatWAfDII49Qu3Zt7r333hPKqCqqip9f8cPOJ0+eXO52MzIy+PXXXwkODub333/3Wu98bm4uAQHVYsEW40u/fgL7foORH0FsT19HYzxARBoD7wMNgHxgoqq+fFIZwZlMfglwGLhBVVd4Opb+cfWZv2EtW9IP0SI61NPVG1Ml+CofORt5cx3oZUBLEWkmIkE4kwRnnlTmd2AAgIjEAcFAuhdjqlCbN2+mffv23HbbbcTHx7Nr1y5uueUWEhISaNeuHY899lhh2V69erFq1Spyc3OpW7cu48ePp2PHjvTo0YO0tOK/T3z22WdcccUVXH311Xz88ceFx3fv3s3QoUPp0KEDHTt2ZMmSJYDzn6Lg2NixYwEYM2YMM2bMKDy3du3aAMyZM4eBAwcycuRIOnfuDMBll11Gly5daNeuHW+//XbhOV9//TXx8fF07NiRQYMGkZeXR4sWLcjMzAQgLy+Pc889t/C+OQvl5cKiZ53e59aX+Doa4zm5wN9UNQ7oDtxRzGTxwUBL1+0W4HVvBDKgTTQAc5OqVP+LMRXC2/lIcT788EPOO+882rdvz4MPPgg4HXLXXntt4fFXXnkFgBdffJG2bdvSsWNHxowZ49kn7yVe61ZU1VwRuROYjbNE3SRVXScijwHLVXUm8DfgLRH5C87wjhtU9YyGaDz61TrW78w60/BP0LZh2GmPq1u/fj2TJ0/mjTfeAOCpp54iPDyc3Nxc+vXrx4gRI2jb9sTPmwMHDtCnTx+eeuop/vrXvzJp0iTGjx9/St1Tp07lySefpE6dOowZM4b77rsPgDvuuIOLLrqIO++8k9zcXA4fPszq1at5+umnWbx4MeHh4W4ls7/88gvr168v7Nl+7733CA8P5/DhwyQkJDB8+HCOHj3K7bffzg8//EDTpk3JzMzE39+fUaNG8dFHH3HnnXcye/Zszj//fMLDw0/rNTTVwK+fQuZWuHoK2OX1akNVdwG7XD8fFJEknPkvRSeLDwXed723/yIidUXkHNe5HtOwbk3izgljblIat/Zp7smqjTktZ1M+crLU1FQeeughli9fTp06dRg4cCD//e9/iYqKYu/evfz6668A7N+/H4BnnnmGbdu2ERQUVHissvPqToSq+o2qtlLV5qr6hOvYw67kGVVdr6o9VbWjqnZS1e+8GY8vNG/enPPPP7/w/tSpU4mPjyc+Pp6kpCTWr19/yjk1a9Zk8ODBAHTp0oWUlJRTyuzYsYPff/+d7t2707ZtW/Ly8khOTgZgwYIF3HrrrQAEBAQQFhbGvHnzuPrqqwuTWHeS2R49epwwLOTFF18s/BaamprKli1b+Pnnn+nXrx9NmzY9od6bbrqJ9957D4BJkyYV9nibs1BB73P986DNEF9HY7xERGKBzsCSkx4qbkJ5oxLquEVElovI8vT08l+MHBgXzfJtmew/fKzc5xpT3XkrHynOkiVL6N+/P5GRkQQGBnLNNdewaNEiWrRowYYNG7jnnnuYPXs2derUAaBdu3aMGTOGKVOmVJnNz6rdwNbKNgO7Vq3/Lam0adMmXn75ZZYuXUrdunUZM2ZMsWsOFh3k7+/vT25u7illPv74YzIyMgoX/D5w4ADTpk3jkUceAU5djkVVi51YExAQQH6+M2czLy/vhLaKxj5nzhwWLVrEL7/8Qs2aNenVqxc5OTkl1hsbG0u9evWYP38+K1euZNCgQcW+PuYssHY6ZG6Bqz6w3udqSkRqA9OBP6vqyV1u7kwodw6e4YTx/m2ieXXeZhZsSOeKzsXm6MZUmLMlHylOSYMJIiIiWLNmDbNmzeKVV15h+vTpTJw4kdmzZ7Nw4UK+/PJLHn/8cdauXYu/v385n2HF8moPtDlRVlYWoaGhhIWFsWvXLmbPnn3adU2dOpU5c+aQkpJCSkoKS5cuZerUqQD069ev8BJNXl4eWVlZDBw4kGnTphUO3Sj4NzY2lsTERAC++OIL8vLyim3vwIEDhIeHU7NmTdatW8eyZcsA6NmzJ/PmzWPbtm0n1AtOL/To0aMZOXJkiZMVTDWXn+f0Pke3gzaX+joa4wUiEoiTPE9R1c+LKeLOhHKP6BhTl8jaQcxNtnHQxpTGk/lIcbp37878+fPJyMggNzeXadOm0adPH9LT01FV/vjHP/Loo4+yYsUK8vLySE1NpX///jz77LOkp6dz+PBhj8bjDdWuB7oyi4+Pp23btrRv355zzz2Xnj1PbyWCLVu2sHv3bhISEgqPtWzZkho1apCYmMiECRO4+eabefPNNwkICODNN9+ka9eu/P3vf6d3794EBATQpUsX3nnnHW699VaGDh3K999/z6BBg6hRo0axbQ4ZMoSJEyfSsWNH2rRpQ7duzpLe9evX5/XXX2fo0KGoKg0bNmTWrFkADBs2jBtvvJEbbrjhtJ6nqQbWfg4Zm+CP74F9iap2XCtsvAMkqeoLJRSbCdzp2kyrG3DA0+OfC/j5Cf1aR/Ptut0cz8sn0N/+5owpjqfykQLvvPMOn332WeH95cuX89hjj9G3b19Ulcsuu4whQ4awYsUKbrrppsKr108//TS5ublcc801HDx4kPz8fO6//35CQyv/SjpyhnP2KlxCQoIuX778hGNJSUnExcX5KCJTkl9++YUHHniA+fPn+ywG+9vwofw8eK0b+AfCbT9VuwRaRBJVNaHsktWXiPQCfgB+xVnGDuBBoAmAqr7hSrIn4CxZehgYq6rLi6nuBMW917vj27W7ue3DRKbe3J0ezSPKfb4xZ8I+c6q24n5/Jb3XWw+08YonnniCiRMnMm3aNF+HYnxl3Reu3ud3q13ybByq+iPFj3EuWkaBOyomIriwZSRB/n7MTdpjCbQxxmvsU814xT/+8Q+2bdtGjx49fB2K8YX8PGfXwag4iBvq62jMWaRWjQC6N49gno2DNsZ4kSXQxhjPWz8D9m6APvdZ77OpcAPaRLN1bzZb0w/5OhRjTDVln2zGGM/Kz3f1PreBtlf4OhpzFurv2pXQeqGNMd5iCbQxxrPWz4D0ZOh9H/hV7nU8TfXUODyE1vVDmZO0x9ehGGOqKUugjTGeU9D7HNkK2g3zdTTmLDYgLpplKfs4cOS4r0MxxlRDlkB7QN++fU9ZhPyll17iT3/6U6nn1a5dG4CdO3cyYsSIEusuaymnl1566YRFxy+55BKP7iXfsWNHRo0a5bH6TDWWNBPSk6D336332fjUgLho8vKVhRvLvyW4MVVVdc1HHnnkEZ577rkzrseTLIH2gFGjRp2yXNu0adPcTjobNmx4wgLk5XXyH+w333xD3bp1T7u+opKSksjPz2fRokVkZ2d7pM7iuLs9qKnECnqfI1pC+yt9HY05y3VqXI/wWkHMs2Ec5ixSnfORysbWgfaAESNG8NBDD3H06FFq1KhBSkoKO3fupFevXhw6dIihQ4eyb98+jh8/zuOPP87QoScu65WSksKll17K2rVrOXLkCGPHjmX9+vXExcVx5MiRwnK33347y5Yt48iRI4wYMYJHH32UV155hZ07d9KvXz8iIyOZP38+sbGxLF++nMjISF544QUmTZoEwLhx4/jzn/9MSkoKgwcPplevXixevJhGjRrx5ZdfUrNmzVOe20cffcS1115LUlISM2fOLPxPuHnzZm677TbS09Px9/fn008/pXnz5jzzzDN88MEH+Pn5MXjwYJ566in69u3Lc889R0JCAnv37iUhIYGUlBTeffddvv76a3JycsjOzmbmzJklvlbvv/8+zz33HCJChw4d+M9//kOHDh3YuHEjgYGBZGVl0aFDBzZt2kRgYKC3ftVn7nAmvDsEjnjuCkGloXlwaA9c+Zb1Phuf8/cT+raOYm5SGrl5+QTYroTmLFCd85HiFFdndnY2V111FampqeTl5fHPf/6Tq6++mvHjxzNz5kwCAgIYNGjQGfdoV78EetZ42P2rZ+tscB4MfqrEhyMiIujatSvffvstQ4cOZdq0aVx99dWICMHBwXzxxReEhYWxd+9eunfvzuWXX46zOdepXn/9dUJCQlizZg1r1qwhPj6+8LEnnniC8PBw8vLyGDBgAGvWrOHuu+/mhRdeYP78+URGRp5QV2JiIpMnT2bJkiWoKt26daNPnz7Uq1ePTZs2MXXqVN566y2uuuoqpk+fzpgxY06J5+OPP+b7779nw4YNTJgwoTCBHj16NOPHj2fYsGHk5OSQn5/PrFmzmDFjBkuWLCEkJITMzMwyX9qff/6ZNWvWEB4eTm5ubrGv1fr163niiSf46aefiIyMJDMzk9DQUPr27cvXX3/NFVdcwbRp0xg+fHjlTp4BdiRC2npocynUrOfraDyvdjS0H+7rKIwBYGBcfT5fsYMVv++na7NwX4djzjaWjxTyRD5yspLq3Lp1Kw0bNuTrr78G4MCBA2RmZvLFF1+QnJyMiHhkWEn1S6B9pOCyScEfbME3IlXlwQcfZNGiRfj5+bFjxw727NlDgwYNiq1n0aJF3H333QB06NCBDh06FD72ySefMHHiRHJzc9m1axfr168/4fGT/fjjjwwbNoxatWoBcOWVV/LDDz9w+eWX06xZMzp16gRAly5dSElJOeX8ZcuWERUVRdOmTYmJieHGG29k3759BAQEsGPHDoYNcyaJBQcHAzBnzhzGjh1LSEgIAOHhZX9gXXTRRYXlSnqt5s2bx4gRIwr/QxaUHzduHM888wxXXHEFkydP5q233iqzPZ9LT3b+vfxVCLEPdGM84lg2/PgSNL0AmvcrPHxhy0gC/YW5SXssgTZnjeqYj5Snzosvvph7772X+++/n0svvZQLL7yQ3NxcgoODGTduHEOGDOHSSy91q43SVL8EupRvZt50xRVX8Ne//pUVK1Zw5MiRwm9qU6ZMIT09ncTERAIDA4mNjSUnJ6fUuor7Nvjbb7/x3HPPsWzZMurVq8cNN9xQZj3ODrrFq1GjRuHP/v7+J1yaKTB16lSSk5OJjY0FICsri+nTp3PVVVeV2F5xsQcEBJCfnw9wSswFf/hQ8mtVUr09e/YkJSWFhQsXkpeXR/v27Ut8vpVGWjLUirbk2RhP8g+ClR9C6rITEujQ4EC6NYtgbnIaD1wS58MAzVnJ8pFCZ5qPlKfOVq1akZiYyDfffMMDDzzAoEGDePjhh1m6dClz585l2rRpTJgwgXnz5rnVTklsUJiH1K5dm759+3LjjTeeMFj/wIEDREdHExgYyPz589m2bVup9fTu3ZspU6YAsHbtWtasWQM4yWutWrWoU6cOe/bsYdasWYXnhIaGcvDgwWLrmjFjBocPHyY7O5svvviCCy+80K3nk5+fz6effsqaNWtISUkhJSWFL7/8kqlTpxIWFkZMTAwzZswA4OjRoxw+fJhBgwYxadKkwgkEBUM4YmNjSUxMBCh1ckJJr9WAAQP45JNPyMjIOKFegOuuu45Ro0YxduxYt56Xz6UnQ3QbX0dhTPXiHwjn3whb50P6hhMe6t8mms1ph9iW4b1J0MZUJtUtHyktvuLq3LlzJyEhIYwZM4Z7772XFStWcOjQIQ4cOMAll1zCSy+9xKpVq86obbAE2qNGjRrF6tWrGTlyZOGx0aNHs3z5chISEpgyZQpt2pSePN1+++0cOnSIDh068Mwzz9C1a1fAWUquc+fOtGvXjhtvvJGePXsWnnPLLbcwePBg+vXrd0Jd8fHx3HDDDXTt2pVu3boxbtw4Onfu7NZzWbRoEY0aNaJRo0aFx3r37s369evZtWsXH3zwAa+88godOnTgggsuYPfu3Vx88cVcfvnlJCQk0KlTp8IB+vfeey+vv/46F1xwAXv37i2xzZJeq3bt2vGPf/yDPn360LFjR/7617+ecM6+ffuqxjJ7qs6He5Ql0MZ4XJex4F8Dlk484fCAOGdXwrlJtiuhOXtUp3ykwOOPP05MTEzhraQ6f/31V7p27UqnTp144okneOihhzh48CCXXnopHTp0oE+fPrz44ovlars4Ulq3emWUkJCgJ69DmJSURFycXZ47G3322Wd8+eWXfPDBB8U+Xqn+Ng6kwovtYMgLcP5Nvo7GnCERSVTVBF/HUV0V915fphl/gnUz4G9JEFyn8PDAFxZSP6wGU8Z193CUxpyoUn3mmHIr7vdX0nu99UCbKuuuuzKqnU0AACAASURBVO5i/Pjx/POf//R1KO4pmEBoPdDGeEfXW+B4NqyccsLhAXHRLNmaycEc25XQGOMZlkCbKuvVV19l8+bNtGrVytehuCfNEmhjvKphJ2jcHZa+Cfl5hYcHtKlPbr6yaGPJQ8iMMaY8qk0CXdWGohjvq3R/E+nJUCsKakX4OhJjqq9ut8K+FNj0feGh+CZ1qRsSyFzbldBUgEr32WPcUt7fW7VIoIODg8nIyLA/WlNIVcnIyChco7pSSE+23mdjvC3uMghtCEveKDwU4O9H31ZRzN+QRl6+fU4Y77F8pGo6nZyhWqwDHRMTQ2pqKunp6b4OxVQiwcHBxMTE+DoMR8EKHB2u9nUkxlRv/oHOJN15/3atetMagAFx9Zmxaicrf99HQqytw268w/KRqqu8OUO1SKADAwNp1qyZr8MwpmQHd8HRrMIPc2OqCxGZBFwKpKnqKbsZiUgd4EOgCc5nznOqOtmrQXW5ARY+4yxpN+R5AHq3iiLAT/g+aY8l0MZrLB85e1SLIRzGVHppSc6/NoTDVD/vAheX8vgdwHpV7Qj0BZ4XkSCvRlQrEs4bAaumwpH9ANSpGUjf1lF8smw7h4/lerV5Y0z1Zwm0MRWhYHe0aFsf1FQvqroIyCytCBAqzp7AtV1lvZ/BFixpt+p/S9rd3rc5+w4f56Mlv3u9eWNM9WYJtDEVIT0JQiKcnjFjzi4TgDhgJ/ArcI+q5nu91YadoEkPZxiHa0m7Lk3D6X5uOG/9sJWjuXllVGCMMSXzagItIheLyAYR2Swi44t5/EURWeW6bRSR/d6MxxifSd8AUdb7bM5KfwBWAQ2BTsAEEQkrrqCI3CIiy0VkuUcmYXW95ZQl7e7s15I9WUf5LDH1zOs3xpy1vJZAi4g/8BowGGgLjBKRtkXLqOpfVLWTqnYCXgU+91Y8xviMqmsJO5tAaM5KY4HP1bEZ+A0odjKAqk5U1QRVTYiKijrzlotZ0q5niwg6Nq7LGwu3kJvn/Y5wY0z15M0e6K7AZlXdqqrHgGnA0FLKjwKmejEeY3zj4G7IOWATCN2Ul698+Mu2Ctt2+cNftvH5CuuN9KLfgQEAIlIfaA1srZCWC5a02zq/cB6CiHBnvxZszzzCzNU7KyQMY0z1480EuhGwvcj9VNexU4hIU6AZMK+Exz17Wc+YipTu2sI72hJod8xN2sNDM9ZWyEQvVWXCvM18t852qDtdIjIV+BloLSKpInKTiNwmIre5ivwbuEBEfgXmAverasXtqd3lBvCv4YyFdhnQJpo2DUL5z4It5NvGKsaY0+DNBFqKOVbSO9VI4DNVLXZWh8cv6xlTkQoSaOuBdkvB2NS5yWleb2vdzix2Z+UwIC7a621VV6o6SlXPUdVAVY1R1XdU9Q1VfcP1+E5VHaSq56lqe1X9sEIDLGZJOz8/4U/9WrA57RDfrd9doeEYY6oHbybQqUDjIvdjcGZhF2ckNnzDVFfpyVAzHGrZl7+yZBw6yrzkNGrXCCBx2z72Hz7m1fbmJachAn1bWwJdrRWzpN2Q886hWWQtJszfbNsuG2PKzZsJ9DKgpYg0cy2aPxKYeXIhEWkN1MO5BGhM9ZO+wel9luIuypiiZq7eSW6+8tCQOPLylYUbvTtka25yGh1j6hIVWsOr7RgfK2ZJO38/4fY+zVm7I8vrf2fGmOrHawm0quYCdwKzgSTgE1VdJyKPicjlRYqOAqapdQGY6kjV2YXQVuBwy/QVqbRrGMZVCY2JqBXE3CTvDeNIO5jD6u37GWjDN84O3W49ZUm7Kzo3omGdYF6bv9l3cRljqiSvrgOtqt+oaitVba6qT7iOPayqM4uUeURVT1kj2phq4VAa5Oy3HQjdkLw7i7U7shjRJQY/P6Ffm2gWbEjz2lJjC5KdXsf+bep7pX5TybS5FMIawY8vOF9sgaAAP27pfS7LUvaxZGuGjwM0xlQlthOhMd6UnuT8az3QZZqemEqAn3B5x4YADIyLJisnl8Rt+7zS3tzkPTSsE0zcOaFeqd9UMv6B0OfvsH0JJP+38PDIrk2IrB3EBOuFNsaUgyXQxniTa+1Z24WwdLl5+Xyxcif920QTUdsZj9yrZRSB/uKV1Thyjufxw6a99I+LRmxs+tmj0xiIbA1zHoE8Z53x4EB/bup1Lj9s2svq7bYZrjHGPZZAG+NN6ckQXBdq2zjb0izalM7eQ0cZ3iWm8FjtGgF0PzeCuUmeX6N5yW+ZHD6WxwAbvnF28Q+Aix6FjM2w4r3Cw2O6NyEsOMDGQhtj3GYJtDHelJZsK3C4YXriDuqFBNLvpOXkBrSJZkt6Nil7sz3a3rykPQQH+tGjeYRH6zVVQKuLoWkvWPAUHD0IQGhwIDf0bMZ36/ewYfdBHwdojKkKLIE2xltUnTHQtgNhqfYfPsb36/cwtFMjggJOfEsqmODnyWEcqsqcpDR6tYgkONDfY/WaKkIELnoMstNh8auFh8deEEtIkD+vL7BeaGNM2SyBNsZbstPhyD7bgbAMX63ZxbG8fEYUGb5RoElECC2jazMv2XPDODbuOcSO/UcYEGfDN85aMV2g3ZVOAp21C4B6tYIY070pM1fvZFuGZ694GGOqH0ugjfEW28LbLdMTU2nTIJR2DcOKfbx/XDRLtmZyMOe4R9qb60rGTx4uYs4yAx52JhIueLLw0LhezQjw9+Ppb5O9tnyiMaZ6sATaGG8pXIHDEuiSbE47xKrt+xkeH1PiahgD4+qTm6/8sGmvR9qcm5RG+0ZhNKgT7JH6TBUV3gy63gwrP3DmKgDRYcHc1a8F3/y6m9FvLyH94FEfB2mMqawsgTbGW9KSoEYdCG3g60gqrekrUvH3E4Z2blhimc6N61I3JJA5HliNIzP7GCt+32erbxhH7/sgKNRZ1s7lrgEtefHqjqxO3c+lr/5A4rZM38VnjKm0LIE2xlvSNzgTCG0FjmLl5Sufr0ilT6sookNL7g0O8Pejb6soFmxIJy9fz6jNBRvSUIUBtn23AQgJhwv/AhtnQcqPhYeHdY7hiz/1JDjQn6vf/IX3FqegemZ/e8aY6sUSaGO8JT3JdiAsxU+b97In6yjD40+dPHiyAXH1ycw+xqoz3OhiblIaUaE1aN+wzhnVY6qRbrc5W3x/9xDk/2/cc9w5Ycy8sxd9W0fxr5nr+MvHqzh8LNeHgRpjKhNLoI3xhuy9cDjD5zsQ5hzP42hunk9jKMn0FamEBQe41Rvcu1UU/n5yRpuqHMvNZ9HGdAa0icbPz64KGJfAmtD/Idi5EtZ/ccJDdWoGMvHaBO4d1IovV+/kyv8s9via5MaYqskSaGO8IS3J+dfHPdDXTVrKHVNW+jSG4mTlHOfbtbu5vFNDt9ZirlMzkPNj6zHvDNaDXp6SycGjufRvY8M3zEk6XA31z4M5j0LuiRMH/fyEO/u35L2xXdmdlcNlE37k+/We3x3TGFO1WAJtjDdUgiXs9mTlsPS3TOYm72HXgSM+i6M436zZxdHcfLeGbxQY0KY+ybsPkrrv8Gm1OScpjaAAP3q1jDyt80015ufvbPG9fxssn1Rskd6tovjvXb1oFlmLm99fzitzN1VwkMaYysQSaGO8IX0D1AiDsJJXl/C2+a7eWlX4YuUOn8VRnM8SU2keVYtOjeu6fU7BUI/5p9ELrarMTd7DBc0jCAkKKPf55izQYgCc2w8WPg1Hih9rH1MvhE9u7cGwzo144fuNLNjguR0yjTFViyXQxnhDerIzfMOHK3DMTU6jUd2aJDStx2eJqZVmFYGUvdks37aP4V1KXvu5OOdG1aZZZC3mJJU/adm6N5ttGYcZYMM3TGkuesxJnn98ocQiwYH+PHnlebSIrs0Dn/9Kloc2+DHGVC1lJtAiUl9E3hGRWa77bUXkJu+HZkwVlp7s0+EbOcfz+HHTXgbERTOiSwxb07PPeAULT/l8RSoiMKxzo3Kf279NND9vySD7aPlWQyiYfNjPEmiPE5FJIpImImtLKdNXRFaJyDoRWViR8ZXLOR2g0zXw82uwZ12JxYID/Xl2RAf2ZOXw5DdJFRigMaaycKcH+l1gNlBwLXoj8GdvBWRMlZedAdnpPk2gf96awZHjefRvE80lHc4hONCP6StSfRZPgfx8ZfqKHfRqEck5dWqW+/wBcdEcy8vnp83l25VwblIabRqEElMvpNxtmjK9C1xc0oMiUhf4D3C5qrYD/lhBcZ2eQY9DcF2YeRfkl7yCTecm9bj5wnOZunQ7P3pol0xjTNXhTgIdqaqfAPkAqpoLVM51sYypDCrBBMJ5SWmEBPnT/dwIwoID+UO7BsxctZOc4779r7s0JZMd+48woov7kweLOj82nNAaAcwtxzCOA4ePs3zbPts8xUtUdRFQ2nZ91wCfq+rvrvKVe+BwSDgMfhp2JMKSN0ot+peLWnFuVC3un76GQ+W8KmKMqdrcSaCzRSQCUAAR6Q4c8GpUxlRlBQl0tG8SaFVlXnIavVpEFi4RNzw+hqyc3HIlnt4wZ/0egvz9GBh3eltpB/r70bt1FPM2pJHv5q6ECzamkZev9Lftu32lFVBPRBaISKKIXFdSQRG5RUSWi8jy9PT0CgzxJO2HQ6uLYd7jsC+lxGIFQzl2HjjCU7NsKIcxZxN3Eui/AjOB5iLyE/A+cJdXozKmKktPhqBQZ3czH0jefZAd+4+c0OPas0UkDcKCfT6MY15yGt2bR1CrxumvhDEwLpr0g0dZu9O97/HzktMIrxVUrhU/jEcFAF2AIcAfgH+KSKviCqrqRFVNUNWEqKioiozxRCIw5AUQf/jqHmcpmxJ0aRrOTT2b8eEvv7O4nEOLjDFVV5kJtKquAPoAFwC3Au1UdY23AzOmyvLxChwFm430a/2/BNrfTxgW34iFG9NJO5jjk7i2ph9i697sM14Jo0+raPwEt3rTc/PyWbAhnX6to/G33Qd9JRX4VlWzVXUvsAjo6OOYylanEVz0CGxdAKs+KrXo3wa1JjYihPs/X1OuCa4Hc46zcGN6pVkhxxjjPndW4bgOZwxbFyAeGFXaJThjznppvl2BY27SHjrG1CE6LPiE48PjY8jLV75cudMncRUk9me6E2B4rSDim9RjbnLZu8ElbtvHgSPHbfyzb30JXCgiASISAnQDqsZ4hy43QpMLYPaDcKjkL2w1g/x5ZkRHUvcd4dnZG8qsVlWZuXonA55fyPWTlvKfBVs8GbUxpgK4M4Tj/CK3C4FHgMu9GJMxVdfhTMhO89kW3nsPHWXl9v3FjvdtEV2bTo3r+mxN6LlJabSuH0rj8DNfCaN/XDRrd2Sx+0DpvenzktMI9BcutN0HvUZEpgI/A61FJFVEbhKR20TkNgBVTQK+BdYAS4G3VbXEJe8qFT8/uPwVOH4Evrmv1KJdm4VzfY9Y3l2cwpKtGSWW25p+iGvfWcrdU1dSPyyYAW2iee67Dcy3TVmMqVLKHIioqieMdxaROsAHXovImKos3dX7FB3nk+YXbEhHlRJ7XId3ieGfM9aybmcW7RvVcbveKUu2sWBDOm+O6YLfaQyFOHDkOMtSMrm597nlPrc4A+Pq88y3G/jDS4sICii5H+DA4eN0axZBaHCgR9o1p1LVUW6UeRZ4tgLC8bzIltDn7zDv35D0X4i7tMSif7+4NfOS0/j79DV8e09vagb5Fz6WczyP1+Zv5s2FW6kR6MdjQ9sxultTjuXmc+Xri7ln6kpm3tmL2MhaFfGsjDFn6HRm8hwGWno6EGOqhXTXlWkf9UDPS95D/bAatGsYVuzjl3doyL+/Ws9nialuJ9DLUzJ5+Mt15OUrv2zN4IIW5e/NXbQxndx89dhOgC2ja3PfH1qTuu9IqeVE4I+nuWSeMYV63gPrZsDXf4PYXlCz+AmpIUEBPD28A6Pe+oVnZ2/g4cvaAs728w/PXMv2zCMM69yIBy5pQ3SoM8SqZpA/E6/twmUTfuSWD5bz+Z96UvsMJtkaYypGmf9LReQrXEvY4Qz5aAt84s2gjKmy0jdAUG2o07jCmz6Wm8+ijXu5rGPDErfIrhMSyEVt6/Plqh08eElcqb23APuyj3HX1JU0qluTfYeP8Vli6mkl0POS06gXEkjnJvXKfW5xRIQ7+rXwSF3GlMk/0BnK8fYA+P5h5+cS9GgewbXdmzJ58W/EN63LV6t3MnvdHppH1eKjm7txQfNT//80Dg9hwqh4rpu0hPs+Xc1/RseXa5t7Y0zFc2cM9HPA867bk0BvVR3v1aiMqarSkiCylU9W4Fj6WyaHjuaW2cs7oksM+w4fL3PMpapy76eryTh0jNeuiefSDucwa+3ucm8YkZuXz/wNabYShqnaGsVDjztgxXvw2w+lFh0/uA2N6tbkzo9WsnBjOvf9oTWz7uldbPJcoFfLSB4YHMestbttUqExVYA7y9gtLHL7SVXdXkhWRC4WkQ0isllEik26ReQqEVkvIutEpPS1goyp7NI3+GwFjrnJe6gR4EfPMnqIL2wZSVRoDaYnlv5f+e0ffmNuchoPXtKG82LqMKJLDEeO5/HNr7vKFdfK7fvZf/g4/W0ljEpNRFqJyFwRWeu630FEHvJ1XJVK3wehXjP46m5nYmEJatUIYMI18Yzu1oTv/9KHO/q1KPNqD8C4C5txeceGNqnQmCqgxP/RInJQRLKKuR0UkayyKhYRf+A1YDDOsI9RItL2pDItgQeAnqraDvjzGT0bY3zpyD44tNsnOxCqKnOT0ujZIvKEiUvFCfD3Y1jnRsxLTiPj0NFiy6z8fR9Pf5vMxe0acP0FsQDEN6lHs8haZSbeJ5ublEaAn9C7lQ83xjDueAvn/fg4gGu9/5E+jaiyCQqBy16GzK0w//9KLdqpcV2eGHZeuVadERGeHt6BNg3CuGfqSlL2Zp9pxMYYLykxgVbVUFUNK+YWqqrFz1A6UVdgs6puVdVjwDRg6EllbgZeU9V9rjbtK7epuvasd/71QQ/0lvRD/J552O01lofHx5Cb76xFe7IDh49z50craVAnmKdHdCgciykiXNm5EUt+y2R75mG3Y5uXvIeuzcIJs5UwKrsQVV160rHyjdc5G5zbB+Kvh8WvQNJXHq++YFKhn59wywfLyz1kyhhTMdwZAw2AiESLSJOCmxunNAK2F7mf6jpWVCuglYj8JCK/iMjFJbR9i4gsF5Hl6enp7oZsTMVaPgkCa0HM+RXedMGufO4m0K0bhNK+URifndSbrKrc99lq9mTlMOGaeOrUPDHpvbJLDCK4vSX49szDbNxz6Iw3TzEVYq+INMc1aVxERgDlG69zthj8NDRKgM9vgZ2rPF59waTCzWmHuO/T1bZToTGVkDs7EV4uIpuA34CFQAowy426i5stdPK7QADOknh9gVHA2yJyyvpAqjpRVRNUNSEqyi4Dm0oofSOsnQ5db4aQ8Apvfm5yGnHnhNGwbk23zxkRH8O6nVkk7/7fiKx3F6fw3fo9jB/chk6NT12qq1HdmvQ4N4LPV+xw60N9bpKzW+DAuFM3djGVzh3Am0AbEdmBM6Tudt+GVEkF1oSRH0HNcJg6ErI8v7unTSo0pnJzpwf630B3YKOqNgMGAD+5cV4qUHQtrxjg5HeZVOBLVT2uqr8BG7A1pk1VtOgZ50P1grvKLuth+w8fI3HbPgaWc5Le5Z0aEegvhWOa16Tu5/++SWJgXDQ39WpW4nnD42P4PfMwy1L2ldnG3OQ0zo2qZZtDVAGu4XYDgSigjar2UtUUH4dVeYXWh2s+hqMHYeooOOb+sCZ3FZ1UOKuck3eNMd7lTgJ9XFUzAD8R8VPV+UAnN85bBrQUkWYiEoQzGWXmSWVmAP0ARCQSZ0jHVrejN6Yy2LvJ6X0+fxzUqvgtoxduTCcvX8s9TCK8VhD920Tzxcqd7Ms+xp0frSSqdg2e+2PHUtegHXxeA2oF+fNZ4vYSywAcOprLkq2ZHts8xXiXiDwsIg8DfwP+UuS+KUmD9jD8bdi1Gr64FfLzPVp9waTC+Cb1uHvaSuYn2zQhYyoLdxLo/SJSG1gETBGRl3FjYomq5gJ3ArOBJOATVV0nIo+JyOWuYrOBDBFZD8wH7nMl68ZUHYuehYBguOBunzQ/NymNyNpBdIwpfne00gyPj2HvoaOMeGMxO/Yf4dVrOlM3JKjUc0KCAhh83jl88+tuDh8r+a3gx03pHMvLZ4AN36gqsovc8nBWUIr1ZUBVQuvBMOhxSJoJ8x/3ePU1g/yZPPZ82jQI49YPE1m8ea/H2zDGlJ87CfRQ4AjwF+BbYAtwmTuVq+o3qtpKVZur6hOuYw+r6kzXz6qqf1XVtqp6nqpOO72nYYyP7N0Mv34KCTdC7Yofn5+bl88C1yYlfqexSUm/NtFE1ApiS3o29/2hNV2aujd+e0SXGA4dzWX2ut0llpmblEZYcABdmnpm90HjXar6fJHbEzhzU06e+G2K0+MOZ2WOH56HVVM9Xn1YcCDv39iVZhG1GPf+cpanZHq8DWNM+ZS2DvQEEblAVbNVNU9Vc1X1PVV9xXqJjXFZ9Cz414Ce9/ik+eXb9pGVk8uA09ykJNDfj3v/0JrrezTllgvPdfu8rrHhxNSryfTEHcU+np+vzN+QRp/W0QT6u73Yj6lcQgD3/yjOZiIw5HmIvdDZZGXbzx5vol6tID4Y15UGYcGMnbyMNan7Pd6GMcZ9pX2ybQKeF5EUEXlaRNwZ92zM2SNjC/z6CZx/E9T2zTjfeclpBPn70avl6fd+j+rahEeHti9XD7afn3BlfAw/bdnLzv2n7si2OnU/ew8dK/fERuM7IvKriKxx3dbhTOp+2ddxVRn+gXDV+1CnMXw8GjJ/83gT0aHBTLm5G3VCArlu0tITVtAxxlSs0jZSeVlVewB9gExgsogkuSaWtKqwCI2prBY9B/5BPhv7DM4ycd3ODad2jYAKb3t4fCNU4YuVp/ZCz0tOw0+gj+0+WJVcijM87zJgENBQVSf4NqQqJiQcrvkE8vPgo6sh54DHmzinTk0+Gted4AB/xry9lK3phzzehjGmbGVeW1XVbar6tKp2Bq4BhuFMCjTm7JW5FdZ87Ix9DvXNJLmUvdlsSc/22SoXTSNq0TU2nOmJqaesCT03KY2EpuFlTkg0vici4SISDhwscjsChLmOm/KIbAFXfwCZW+DTsZDn+Z0Em0SE8OG4bqgqo99eUq6dQY0xnuHORiqBInKZiEzB2UBlIzDc65EZU5ktet65ZOujsc/grLEM0L+N71a5GNElhq17s1m5/X/jMXfuP8L6XVmnPS7bVLhEYLnr35Nvy30YV9XVrDcMeQG2zIVv7gUv7CTYIro2H47rxuFjeVzz9i/sOnDqUCpjjPeUNonwIhGZhLPZyS3AN0BzVb1aVWdUVIDGVDqZW2H1VOgyFkIb+CyMecl7aBldmyYRIT6LYfB5DQgO9DthS/B5rsTeEuiqQVWbqeq5rn9PvtkkwtPV5Xro+WdInAyLX/FKE3HnhPH+jV3Zl32c0W8vYe+ho15pxxhzqtJ6oB8EfgbiVPUyVZ2iqtkVFJcxldcPz4NfgE97nzenHWLxlgwGt/ddAg8QGhzIxe0a8NXqneQczwOcBLpJeAjNo2r7NDZTfiJST0S6ikjvgpuvY6rSBvwL2g2D7x+Gdd7pd+rYuC6Tx55P6r4jPPrVeq+0YYw5VWmTCPup6luqagtOGlNgXwqsngZdboCwc3wWxqvzNhEc4M/1F8T6LIYCI7o05mBOLt+v38ORY3n8tHkvA+KiS93N0FQ+IjIOZ8Os2cCjrn8f8WVMVZ6fH1zxOsR0dXYq3L7MK82cHxvO7X2a89XqnSzeYhutGFMRbIFWY8rjh+dB/KDXn30Wwpb0Q3y1eifX9WhKRO0aPoujQI/mEZxTJ5jpK1L5afNejubmM8CH47LNabsHOB/Ypqr9gM5AelknicgkEUkTkbVllDtfRPJEZIRnwq0iAmvCqKnOcK+pI72yvB3A7X2b0zi8Jv/6ch3H8zy7pbgx5lSWQBvjrn3bYNVHrt7nhj4LY8K8zdQI8Ofm3pVjeKq/nzCscyMWbUxn2rLfqV0jgK7NbPGGKihHVXMARKSGqiYDrd04713g4tIKiIg/8DROr/bZp1YkjP4M8nPho6vgyD6PNxEc6M/Dl7ZjU9oh3luc4vH6jTEncmcVjjtFxPbiNaag97mn73qft6Yf4stVO7i2R1MiK0Hvc4HhXWLIV5iTlEbvVpEEBdh38yooVUTqAjOA70XkS2BnWSep6iKcvQJKcxcwHUg74yirqsiWMHKK0wP98bWQe8zjTQyMi6Zv6yhemrOJtKwcj9dvjPkfdz7lGgDLROQTEblYbGCjORvt/x1WTYH466BOI5+FMWH+ZoIC/Li5HNtuV4TmUbXp3KQu4Ntl9czpU9VhqrpfVR8B/gm8A1xxpvWKSCOc/QPecKPsLSKyXESWp6eXOXqk6ontBUNfg5QfYOZdHl/eTkR45LJ2HMvN58lZyR6t2xhzInc2UnkIaInzZnoDsElE/k9Emns5NmMqj3VfOJdffdj7/NvebGas3MGYbk2JCq08vc8Fru8RS2iNAPq1tt0HqxIR+VpERotIrYJjqrpQVWeqqie6SV8C7lfVvLIKqupEVU1Q1YSoqGr6d9Txauj7IKyZBguf9nj1sZG1uKX3/7N33+FRldkDx79vKqR3CEkgCQkQIBB6EZSmgKggKkhT7B3Xsq7uz3Vd3bXt2kERGyoiiiJYUJEOIp3QkgChpRCSUEJCenl/f9yJBkiZSWZSz+d55oFM7tx7CHBzcua85w3n212pbD0qMwCEsBWz3mfVxjZjJ02PEsAb+Fop9YoNYxOi8chIAPdA8AppsBBmr07E0d6Ou69oXNXnchN6BbHzmSsbxcJGYZF5GNt4H1NKfamUmqCUsuYWkn2BRUqpY8CNwDtKqTpXtpu0K56AnlNh7YvGVB8re2B4BEFerXlm2T5KivwVswAAIABJREFUZEGhEDZhTg/0LKXUDuAV4DcgWmt9H9AH2ZFQtBSZ8eBvznoq2zh+OpelsalMH9iBAPdWDRZHTRztpfe5qdFaL9NaTwHaA0uAW4Ek03SNK61w/jCtdajWOhT4Gri/xW/GpRRc+yaEDoVlD0LiSquevrWTPf+4JoqEkzl8tvm4Vc8thDCY893OD5iotR6ttV6stS4G0FqXYVQthGjeysog8yD4RzVYCLNXJ+Jgp7inkVafRdOntc7XWn+ptb4euApjjN3PNb1OKfUFxqZbnZVSKUqpO5RS9yql7rVxyE2bgxNMXgD+XWDRdDi+yaqnH92tLUMj/XhtxUEyc2SHQiGszZwEejkVVlgrpdyVUgMAtNbxtgpMiEYjOwWKcxusAp10Oo8lu1KZOqB9o64+i6ZNKdVGKfWQUuo3jEkcKzDeaayW1nqK1jpQa+2otQ7WWn+otZ6rtb5k0aDWeqbW+msbhN80tfaCGd+CZzB8PglSd1rt1Eopnr2uGwUlpbwkCwqFsDpzEuh3gfMVPs41PSdEy5Bh+ubj36VBLj97zSHs7RT3XiHrdoX1KaXuUkqtBnYCnYAntNbhWuu/aa1jGzi85s/NH25ZBi7esGAipFtvO+6O/m7cMSScb3amsOO4LCgUwprMSaCVaREh8EfrhoPtQhKikcksT6DrvwKdfCaPJTtTmdq/PW08pPosbGIw8BIQorV+SGv9W0MH1OJ4BhlJtL0zfDYBTh+22qkfGhFBoGcrnlm2n9Iy647NE6IlMyeBPmJaSOhoejwMHLF1YEI0GpkJ4NYGXOp/d705axKxs1PcN0yqz8I2tNa3aa1XmIojoqH4hBtJdFkJfDoespKtclpXZwf+b1wU+09ks3CLLCgUwlrMSaDvxahQpAIpwADgblsGJUSjkpnQIO0byWfy+HpHClP6hUj1WYiWIKALTF8CBdlGEp2TbpXTjosOZHBHX/77ywFOn5cFhUJYgzkbqWRorW/WWgdordtoradqrVvudqyiZdEaMg80SAL9ztrD2CnFvVJ9FqLlaBcD0xZDzkmjnSOv7r3LSin+dV038opKeWrJXsqklUOIOjNnDnQrpdQDSql3THNBP1JKfVQfwQnR4M6lQNH5eu9/Tjmbx+LtyUzuF0KgZ+t6vbZomZRSHZVSzqbfDzO17nk1dFwtUvsBMGWh0Qu9YKJRka6jyDbuPDm2Cyvi0nl7daIVghSiZTOnheMzoC0wGlgHBAM5tgxKCHPkFZVQUFzj7sB1k3nA+DWgfmdAl1efpfdZ1KNvgFKlVATwIRAGLGzYkFqw8GEw6RM4uRcWToai3Dqf8o4hYUzsHcTrKw/yy/6TdT6fEC2ZOQl0hNb6H0Cu1voTYBwQbduwhKheaZlm4jubeOyr3ba9UKZp1Hk9tXAUlZTx6ooDfLktmUn9gmnnJdVnUW/KtNYlwPXAG1rrR4DABo6pZes8FibOg+TNRk90Hds5lFK8cH00PYM9efTLWA6clFqYELVlTgJdbPo1SynVHfAEQm0WkRBm+GHPCRJO5rD+YKZtRzNlJoBrQL1M4NiXeo7rZm/k7dWJTIgJ4smxDbfzoWiRipVSUzC28v7B9JxjA8YjALrfADd9Aml74KPRdZ7O0crRnvdm9MXF2YG7Pt1OVl6RlQIVomUxJ4Gep5TyBp4GvgPigJdtGpUQ1Sgt07y9OhFHe0VOYQlxJ+reH1ilzAM2738uLi3jjZUHmTDnN87kFvHhrX15dVJP3Jxl3LqoV7cBg4D/aK2PKqXCgAUNHJMA6HodzFhiLCz8aDRk1G0T4LaerZg7vQ8nzxXw0Be7KCmVCYZCWKraBFopZQdka63Paq3Xm3anCtBav2fOyZVSY5RSB5RSiUqpJyv5/EylVKZSKtb0uLOWfw7Rgizfm0Zixnn+NsZoq9h85LRtLlQPEzjiTmQzfvZvvLHyENf2bMeKRy5nZFQbm11PiKporeO01rO01l+YiibuWuuXGjouYRI6BG5bbsyJ/mgMJG2p0+n6dPDm39d3Z8OhU7LVtxC1UG0CbRqs/2BtTqyUsgfmAGOBrsAUpVTXSg79UmsdY3p8UJtriZajrEzz9upDRAS4cdtlYYT5ubLlqI0S6OwTUJhtzGa1suLSMt5edYjxczaSkVPIvBl9eH1yDF4uTla/lhDmUEqtVUp5KKV8gN3Ax0qp1xo6LlFB22i4Y4XRUvbpeDjwc51ON6lvCDMHh/LBxqN8syPFSkEK0TKY08Lxq1LqcaVUiFLKp/xhxuv6A4la6yNa6yJgETC+TtGKFu+nfSc5mH6eh0ZEYG+nGBDmw5ajZ2zTB33RAsIKO9rXyYGTOUx8ZxOv/nqQsd0D+fWRy7mqW1urnFuIOvDUWmcDE4GPtdZ9gFENHJO4mHco3L7CaC1bNBV2fV6n0/3fuCgGd/TlqW/3EpucZZ0YhWgBzEmgbwceANYDO0yP7Wa8LgiouNohxfTcxW5QSu1RSn2tlAox47yihSor07y56iAd/V25pkc7AAaG+5JTUEJ8mg36oMtH2PlH8eOeNPr9ZyUZ2QV1OmViRg7Xzt7Iiax83p3Wm7em9MLbVarOolFwUEoFApP4cxGhaIzc/GHmDxA2FJbdDxtfN1rOasHR3o45U3vTxsOZez7bXud7XLmzuUXsSz1nlXMJ0RiZsxNhWCWPcDPOrSo73UUffw+Eaq17ACuBTyo9kVJ3K6W2K6W2Z2ZmmnFp0Rz9vN+oPs8aGYm9nfHPa0C48WaITfqgMxPAxQ9cffl2Vwqnzhcxd92ROp3yzVWJONoplj88lLHRMiFMNCrPAb8Ah7XW25RS4cChBo5JVMXZHaYuNqZ0rHwWfvk/KKvdYkBvVyfmzehLdn4J9yzYQWFJ3efrP/pVLBPm/CZVbdFsmbMT4S2VPcw4dwpQsaIcDJyoeIDW+rTWutD04ftAn8pOpLWep7Xuq7Xu6+/vb8alRXNTVqZ5a9UhwitUnwECPVvTwdeFzUfqvt3tJTISwL8LBcWlbEw8haO94vMtx8nIqV2FJjEjhx/2nOCWwaG08Whl5WCFqBut9WKtdQ+t9X2mj49orW9o6LhENRycYOIH0P8e2DwHvppR610LowI9eG1ST3YlZfHKzwfqFNaxU7msOZBJqdbM+mIXOQXFNb9IiCbGnBaOfhUeQ4FngevMeN02IFIpFaaUcgJuxhiD9wfT24XlrgPqNptHNFsr4k6ScDLnj97nigaG+bLt2BnKrNkHXT6BI6ALvx8+TUFxGf+4pislZZp5taxCv706kdaO9tw11Jw3cISoX0qpYKXUt0qpDKVUulLqG6VUcEPHJWpgZwdjX4bRL8KBn+D9EX+2n1lobHQgN/UJZsHm45w+X1jzC6rw6e/HcbBTvDO1N6lZ+fxj6b5an0uIxsqcFo6HKjzuAnoBNTZtmna0ehDjLcF44Cut9X6l1HNKqfIEfJZSar9SajcwC5hZ2z+IaL6M3udEwvxcubZC9bncgHAfzuUXE3/Sin3QOWlQeA78u7AqIR0XJ3sm9Q1hfEw7Fmw5TmaOZd9cEjPO893uE8wY1AEf6XkWjdPHGEWOdhjrVb43PScaO6Vg0P1w63dQkGUk0XHLanWqe64Ip7CkjM82H6/V63MLS1i8I5mx0YGMjQ7k4ZGRLI09wZKdMuVDNC/mVKAvlgdEmnOg1nq51rqT1rqj1vo/puee0Vp/Z/r9U1rrblrrnlrr4VprGUYpLvFrfDrxadk8ODwCB/tL/8kOCPcFYIs12zgyjX+K2r8zq+MzGBLhRytHex4aEUlRSRnz1h+26HSzVx+ilYM9d0v1WTRe/lrrj7XWJabHfEB65pqS0CFw9zpjctBXt8Cv/4Qyy/qZIwLcGRUVwKe/H6eg2PJe6KWxqeQUlDBzcAcAHhgeQf8wH/6xdB/HTuVafD4hGitzeqC/V0p9Z3r8ABwAavejrRAW0lrz5spDhPq6MD7m0uozQJBXa0J8Wlt3IaHpLdCDZcGcOFfAKNPmJmF+rkyICeKzzcc5ZeZbnEcyjerzLYM64OvmbL0YhbCuU0qp6Uope9NjOlDjfyql1Eemto9K36dXSk0zTVrao5TapJTqafXIxZ88g4wNV/rcBr+9AQsmQq5l98a7hoZzJreIry2cDa215pNNx+jWzoPe7b0BsLdTvDE5Bgd7O2Yt2kVRiex6KJoHcyrQ/wNeNT1eBC7XWl+yq6AQtvBrXDpxadk8OCKy0upzuYFhvmy1Zh90Rjy09uHXYyUADOvyZyHuwRERFJWU8f5683qhZ69OxNnBnrsul+qzaNRuxxhhdxJIA27E2N67JvOBMdV8/ihwhWna0vPAvLqFKWrk4AzXvgHXvQ3HN8G8YXAi1uyX9w/zoWeIFx9uPGrRjP3NR85wMP08tw4ORak/16q082rNyzf0YE/KOV79tW4LFIVoLMxJoJOALVrrdVrr34DTSqlQm0YlBKbq86pDdPB1YUIV1edyA8N9ycor5kB6jnUunnkAAqJYmZBJz2BPAtz/nJoR7u/GdT3b8envNVehj57KZWlsKtMHtsdPqs+iEdNaJ2mtr9Na+2utA7TWEzA2VanpdeuBKvuntNabtNZnTR9uxpjIJOpD71vg9p9Bl8JHoyF2oVkvU0px99Bwjp7KZWV8utmX+2TTMbxdHLmu56X36zHd2zJtQHveW3eEDYdkHK1o+sxJoBcDFd9zKTU9J4RNrYrPYP+JbB6oove5IqvOg9YaMuPJ94pgd0oWI03tGxU9OCKSgpJS3t9QfRV69upEnBzsuPvyjnWPS4j696iVz3cH8JOVzymqE9TH6IsO7gdL74NlD0JhzYWG0d3aEOLTmnlmvtOWmpXPiriTTO7XnlaO9pUe8/S4rkQGuPHoV7vNboETorEyJ4F2MG3FDYDp9zJGQNhUefU5xKc11/eqbAPLCwV7uxDs3do6CwnPp0PBOeJLgtAaRnQJuOSQiACjCv3Z78c5k1tUyUmMWahLY1OZNqAD/u5SfRZNUmUbYtXuREoNx0ig/1bNMbJpli24+cOMpTDkUYj9HN4dDMd+q/YlDvZ23DkknB3Hz7LjeM331c9NUzumDWhf5TGtnex5e2ovzuUX89fFu9G13D1RiMbAnAQ6s8LYOZRS44FTtgtJCFhzIIO9qed4cHgEjjVUn8sNCPNly9HTde+DNk3gWH/Wh7YerejWzqPSwx4aEUF+cdVV6NlrEnGwU9xzhfQ+iybLKhmOUqoH8AEwXmtd5dtEsmmWDdk7wKh/wm0/gbKD+eOM3QuLq94Y6qa+wXi2duT99UerPXVBcSmLtiUzMqoNIT4u1R7bpa0HT4+LYs2BTD7+7Vht/iRCNArmZCb3An9XSiUppZIwqgf32DYs0dK9vTqRYO/WTOxtfrvkwHAfzuYVcyjjfN0unmEk0N+muDMiKuCCxTAVRQS4c02Pdnyy6dglVejjp3P5dlcqUwe0v6B/WojGRimVo5TKruSRgzETuq7nbw8sAWZorQ/WOWBRN+0Hwr2/Qd/b4PfZ1S4wdHFyYMbADvwSd5Kj1Yyg+3FPGmdyi5g5ONSsEGYM7MCoqDa89FMC+1LP1eIPIUTDM2cjlcNa64FAV6Cb1nqw1jrR9qGJlup8YQm7krK4qU+I2dVnMBYSghX6oDMTKHby4niRGyMrad+oaJapCv3BRVXoOabq831XSO+zaNy01u5aa49KHu5aa4eaXq+U+gL4HeislEpRSt2hlLpXKXWv6ZBnAF/gHaVUrFJquw3/OMIczm5wzesw7WvIPwsfjIR1/4XSkksOvXVwKI52dny4sfJ32rTWfPL7MSIC3Bjc0desyyuleOXGHni7OjLri12cy5OtvkXTY84c6BeUUl5a6/Na6xyllLdS6t/1EZxomRLSjB0FuwdV3jpRlRAfF4K8rDAPOjOBE44dcHawZ3BHv2oPjWzjztXRgXyy6RhnTVXo5DN5LNmZypT+7QnwkOqzaN601lO01oFaa0etdbDW+kOt9Vyt9VzT5+/UWntrrWNMj74NHbMwibwS7v8duo6HNf82JnWcurA+5u/uzMTeQSzenlLp9t6xyVnsSTnHrYM6VPluXWV8XJ146+ZepJzN57b5W8krujR5F6IxM6e8N1ZrnVX+gWkc0dW2C0m0dHGmBDoq0LIEGoxpHFuOnqn94hSt0Rnx7Cpsw5AIP1o7Vb6avKJZIyLJKy7lw41Gn+CcNYnY2SnuGybVZyFEI+fiAzd+BDd8CKcTYe4Q2DwXyv4cvnXn0LAqt/f+ZNMx3JwduN6CdrtyA8J9eWtKDLHJWdzz2Q4KSyzf+VCIhmJOAm2vlPpjhIBSqjUgIwWEzcSnZePl4kigp+XV24FhvpzJLap9H3RuJqogi9j8toyIqr59o1zntu5c3T2Q+ZuOsS/1HF/vSGFKvxDaSPVZCNFURN8I9282tgP/+W8w/+o/qtFVbe+dmVPIj3vTuLFPMG7ONXb7VGpM90BemtiDDYdO8eiXuy3auEWIhmROAr0AWGXqa7sd+BX41LZhiZYs7kQ2XQM9LHo7sFx5H/SW2rZxZMQDcEgHVTq+rioPjYzgfGEJ0z/cgp1S3CvVZyFEU+MRCNMWw4R3ISMO5l4Gv70JpSV/bO/9zc4/t/f+YmsSxaWaGYM61Omyk/qF8PS4KH7cm8bTS/dabbxdenYBK+PSee3Xg9w+fxsj/reWL7YmWeXcQtT4I6PW+hWl1B5gFMZM0Oe11r/YPDLR5JzLK+b1lQe5f1jHWvf+lpSWkXAyh+kDa3dDDvFpTaBnKzYfOcOMQaGWnyDTtM2sfxSBnq3NflmXth6M7d6Wn/adZMbADha9VgghGg2lIGYqdBwBPz4Gvz4D+5fSf/xsegZ78sGGo9zcrz1lWvP5luMMjfSjo79bnS9759BwsvKKmb0mEc/WTjw5totFr8/MKWRPitGPvS/1HHtSz5GZY/Rs2yljdn9rJ3ueWrKXkjLNjFp+jxGinFnvuWitfwZ+BlBKXaaUmqO1fsCmkYkmRWvNX7/ezYq4dEJ9XZh5WVitznPsdC6FJWV0rUX/MxiruweG+7LhUCZaa4ur2IUn9lOgXejTtbPF1358dGdyCkp4YHiExa8VQohGxb0tTF4A+7+F5Y+j3ruCVzrfx7hdfVkZn05JqSY9u5AXro+22iUfu6oTWflFzF13GM/WjjWuI9Fas+3YWeatP8KqhHS0NvL/jv5uDI3wo3uQJz2CPenazgMXJwcKS0p54POd/GPpPgBJokWdmJVAK6VigCnAZOAoxkxPIf4wf9MxVsSlo5SxKru29p8wFhB2rWLzEnMMDPfh212pHM48T0SAu0WvzUnZx1EdzIiubS2+bkd/NxbcOcDi1wkhRKOkFHSfCGGXw09/o/O+t/ipdSjvrXqMJKdIQnxaM6yz+a1uNV9O8dx13cnOL+HlnxPwcnFkSv9LdzYsKS3j5/0neX/9EXannMPbxZEHhkVweSd/urXzwLWKfmxnB3vmTOv9ZxKtde3eqRSCahJopVQn4GaMxPk08CWgtNbD6yk20UTsScniheXxjOwSgJ2dYndK7Qfjx6Vl42RvV6e3BAeEGX3Qvx85Y3EC7Zx1iGT7fkwI8qz19YUQollx9YMbP4TuEwla8jAvnX6YD0rH0WrEE9jbWW23dwDs7BSvTupJTkExf/92Lx6tHBnXIxAw9gj4alsyH/12lJSz+YT5ufLvCd25oXewWROT4KIketl+AEmiRa1UV4FOADYA15ZvnKKUeqReohJNRnZBMQ8u3IW/mzP/u6knX2xL4te4dLLyivBycbL4fPFpOUQEuOHkYP4GKhfr4OtCW49WbDly2qK36Iqz03EvPYdD2y7YWfmbghBCNHldxsEDA/j+9Tu51+F7yrZvBs+nodcMsDMvgTWHo70d70zrwy0fbeEvX+6ipKyM+LQcFm45TnZBCf1CvfnHNV0ZFdWmVgm8JNHCGqrLUm4ATgJrlFLvK6VGYiwiFAIw+s+e+mYvqVn5vD21F96uTsQEewHUugoddyK7Tu0bYLwNOCDch81HLJsHfXCvsUFaYGTvOl1fCCGaKxdPP1rd8C6rhi7Czi8Cvn8Y5g6Fw2usep3WTvZ8cGs/IgPceXhRLPPWH2ZopD/f3j+YxfcOZnS3tnWqfpcn0aOiAvjHsv18+vsxq8UuWoYqE2it9bda68lAF2At8AjQRin1rlLqqnqKTzRiC7Yk8ePeNB6/qjN9OvgAEB3safRBJ1neB52RU8Cp84W1XkBY0cBwX06dL+TIqVyzX5N8YAcA3Xr2q/P1hRCiuRobHcjIkWPhtp/gpk+g6Dx8NgEWTobMg1a7jmdrRz69oz9Pje3Cur8OZ8603vRq72218zs72PPOtD6MigrgGUmihYVqfJ9ca52rtf5ca30NEAzEAk/aPDLRqO0/cY7nf4jjik7+3HN5+B/Pu7dyJMLfjd0plifQcSdqvwPhxcrnQVuyrXdBWhy5yhUX35A6X18IIZo9paDbBHhgK1z5HBzfBO8OguVPQN4Zq1zCz82Ze67oSIiPi1XOdzEnBztJokWtWNRoqrU+o7V+T2s9wlYBicbvfGEJDy7chbeLI69N6nlJv3BMiBexyVkWD8OPT8sBsEoFOtTXhQB3ZzYfMe8mfiTzPG0KjpPrEWF8UxBCCGEex1Zw2cPw0E7ofQtsex/e6gWb34XSkoaOrkYXJ9Gvrjgg24qLGtV+pZZokbTW/N+3ezl+Ope3bu6Fr9ulu7r3DPHiTG4RKWfzLTp3XFo2QV6t8XRxrHOc5fOgtxw5bVYivzohg0i7FFyCutX52kII0SK5+cM1r8N9myCoN/z8JMy7ApK2NHRkNSpPoif2CuLt1YmMe2sj249Zp4oumidJoIVFvtyWzLLYEzwyqhMDTG0SF4sJMRYS7rJwHnTciXNWad8oNyDch4ycQo6a0Qe9bf9B/FQ2biHdrXZ9IYRokQKiYPoSYyOW/LPw0VWw7EHINb+lriE4Odjx2uQYPp7Zj/yiUm6c+ztPL91LTkFxQ4cmGiFJoIXZEk5m88/v9jMkwo/7q9ltr3Nbd5wd7NhtQQKdX1TK0VO5dZ7AUVF5H/SWo9VXEbILislJNnamwt/yHQiFEEJcRCmIutbojx48C3Z/AbP7wI5PoKysoaOr1vAuAax45HJuuyyUz7ckceVr61mx/2RDhyUaGbN2IhQir6iEBz7fiUdrR16fHFPt+CBHezuigzwt2pHwQHoOZdo6/c/lwv1c8XNz5psdKRQWV93Pdjgzl3BSjA/8o6x2fSGEaPGc3eCq56HnFPjxMfh+FuxaANe8Bm2ttw24tbk6O/DPa7sxPiaIJ7/Zw92f7eDq6LY8e203AjxaNXR4ohGQBFqY5bvYExzOzOXT2/vj735p3/PFeoZ4sWDzcYpLy3C0r/mNjvIJHN2sWIFWSnFl1zZ8sTWJ7cfPVnvs/1zT0fbuKI92Vru+EEIIkzZd4bblsHsRrHga3rscBtwLw56CVta771tbTIgX3z80hHnrj/DmqkNsOHSKv18dxeS+IbLhVgsnCbQwy8r4DIK8WjM00s+s42NCvPhw41EOnMyhuxnbYsenZePu7ECwd+u6hnqBF67vzhOja27L8PzqXVRJZ5nAIYQQtqIUxEyBTqNh9fPGlI69i6HfXdD3dmMRYiPkaG/HA8MjuDo6kKeW7OGpJXvZfuwsr07q2dChiQYkPdCiRgXFpfyWeIqRUQEoMxPM8oWE5rZxxKVlExXoYfb5zaWUwtvVqcaH3akDENDFqtcWQghRCRcfY1rHnasgMAbWvgCvd4OlD8DJvQ0dXZXC/Fz54q6B3D+sI9/sTOGHPScaOiTRgGyaQCulxiilDiilEpVSVW6+opS6USmllVJ9bRmPqJ3fj5wmv7iUkVFtzH5NsHdrfF2dzEqgy8o08Wl138K71vLOQG4G+EsCLYQQ9Sa4D0z/Gh7YBr2mw/4lMHcIzL8GEn6EssY3i1kpxaNXdqJniBdPL91HRnZBQ4ckGojNEmillD0wBxgLdAWmKKW6VnKcOzALaPyDIluoVfHpuDjZMyDMx+zXKKXoGeJl1iSOpDN55BWVEhXoXpcway8zwfhVFhAKYTGl1EdKqQyl1L4qPq+UUm+ZCil7lFK96ztG0cj5dzIWFT6yH0b9C84chUVT4e3eRptHQXZDR3gBB3s7Xr2pJ/lFpTy5ZK/Fm4aJ5sGWFej+QKLW+ojWughYBIyv5LjngVcA+TGuEdJaszo+gyERfrRytLfotTEhXiRmnq9xhmZcmnFz7BpYc6+0xXLSIWF59Y/9S41jZYSdELUxHxhTzefHApGmx93Au/UQk2iKXHxgyF/g4d1w03xwa2NsxvJaV/j5KSOxbiQiAtx4cmwXVidk8NX25IYORzQAWy4iDAIq/qtKAQZUPEAp1QsI0Vr/oJR6vKoTKaXuxrjx0r59exuEKqqScDKHE+cK+MuoTha/tmeIF1rD3pRzDI6oevFh3Ils7O0UkW3c6hLqpbSGRVMgdUfNx7r4gWewda8vRAugtV6vlAqt5pDxwKfaKNNtVkp5KaUCtdZp9RKgaHrsHaDb9cYjdYdRhd46z/i1yzgYeD90GNzgi75vHRTKiv3pPPd9HIM7+hHi49Kg8Yj6ZcsEurJ/2X+8z6GUsgNeB2bWdCKt9TxgHkDfvn3lvZJ6tCo+HYBhXSxfHR0T/OeOhNUl0PFp2UT4u1lc4a5R4krj5jvyGeg4svpj3QMb/GYsRDNVWTElCLgkgZZiibhEUB+44QO48jnY9gFs/wgSfoC2PYxEuvtEcKh5tKot2Nkp/ntTD8a8sYHHF+/mi7sGymi7FsSWLRwpQEiFj4OBiktW3YHuwFql1DFgIPCdLCRsXFYlZNAz2JMAd8sHx3u6OBLu51pjH7QxgcPK/c9aw9oXwbM9DHoI2sVU/3A3f4GkEMKvGg/lAAAgAElEQVQi1RZTLnhS63la675a677+/o1zpJloIB7tjGLII3Fw7ZtQWgRL74U3omHdK3A+s0HCCvZ24Zlru7Ll6Bk+3nSsQWIQDcOWCfQ2IFIpFaaUcgJuBr4r/6TW+pzW2k9rHaq1DgU2A9dprbfbMCZhgVPnC4lNzmJEl9onlz1DvIhNzqpykcWZ3CLSzhVYfwJH4iqj+jz0UXBwsu65hRCWqKmYIoT5nFygz0y4fzNMX2JUotf8xxiD9/UdcGRtvW8VflOfYEZFBfDKzwkkZpw3+3Vaa8rK5E31pspmCbTWugR4EPgFiAe+0lrvV0o9p5S6zlbXFdazJiEDrWFkVECtzxET4kVGTiEnqxj1E2+LBYRaw7qXwDMEYqZZ77xCiNr4DrjFNI1jIHBO+p9FnSkFESP/HIPX+xZI/BU+HQ9vxRhV6XMp9RSK4oWJ0bg42fPYV7GUlFafwJeUlrF0VypXvr6e6Gd/4cWf4snMKayXWOsi+Uwe//0lgaGvrOa+BTtIb+Ej/Gy6E6HWejmw/KLnnqni2GG2jEVYbnVCBm08nOu0vXbP8g1VkrIIjL50l8HyBNqqLRyHV0PKNmNQv1SfhbAppdQXwDDATymVAvwTcATQWs/F+B5wNZAI5AG3NUykotny7wTj/gdXPQ/xP8CuT42q9JoXjCS71wzofLVNvx8EuLfiP9dHc//nO3ln7WFmjYy85Jji0jK+3ZXKO2sSOXY6j05t3Liisz/vrz/C/N+OMblfCPdc0ZEgL+vuyFsXJaVlrErIYOGWJNYfykQBA8N9WZ2QwcbElr2tuWzlLSpVVFLG+oOZXBcTVKfdAaMC3XGytyM2JYux0YGXfD7uRDZtPJzxdbPSIhCtYd3L4BEMMdOtc04hRJW01lNq+LwGHqincERL5tgaetxkPM4chdiFEPs5LL4VXHyhx83Q51abjSy9OjqQ8THteGvVIYZ3DiA62HhntbCklMXbU3h37WFSs/Lp1s6DudP7cFXXNtjZKY6eymXu2sN8sTWJhVuSuL5XEPcO60hHfytPprJAalY+X25N4svtyaRnF9LGw5mHRkRyc78Q2nm15tipXJ5aspenluxlWWwqL07sQZifa4PF2xAkgRaV2nr0DLlFpYyqQ/sGgLODPVHtPIhNqnwhYVxaNl0Drdj/fGQNJG+Bca9K9VkIIVoqnzAY8X8w7Ek4vMaoSm+dB5vnQIch0Pc2iLrW6hM8nruuO5uPnObRr2L5+t7BfLMzhffWHyY9u5Be7b3494TuDOvsf0FhKszPlZdv7MHDoyKZt/4Ii7Yl8fXOFK6ODuT+YR3p1s4GeyRUYf3BTOZvOsbaAxlo4IpO/jw/vj0jugTgYP9n12+onysL7xrAl9uS+c/yeMa8sZ6/jOrEnUPDcLS36SbXjYYk0KJSK+PTcXawY3DHqsfPmatXiBdfbU+mtExjX+FtnsKSUhIzzjOiS92S9D9oDWtfBo8g4y07IYQQLZudPUSOMh7nM42K9I6P4Zs7jPn/vaYbixJ9wqxyOU8XR16+oQczP95GvxdWUlRSxoAwH16bFMPgjr7VvqPbzqs1z17XjQdHRPDRxqN89vtxftyTxrjoQN68OeaCBNYWft6Xxr0LduLv7sz9wyKY3C+k2tnWSilu7m8k188s28/LPyfww54TvHxDD7oH1V/S31Baxo8JwiJaa1YlpHNZhB+tneo+m7lniCd5RaUcysi54PlD6ecpKdPWm8BxdB0kb4YhjzTYXFAhhBCNlJu/sdPhQ7tg+jfQfiBsettYdPjZRKN/urSkzpcZ1jmAWSMiGBrhx1f3DOLLewZxWYSf2e2Qfm7OPDGmCxufHMHdl4fz4940Vpr2ZLCVI5nneXzxHnqGeLHhieE8Prqz2RvDBHi0Yu6MPsyd3puMnELGz/mNF3+KJ7+o1KYxNzSpQItLHM48T/KZfO69oqNVzhcT4g3A7uQsurT9M1n+cwtvKyTQ5dVn93bGamwhhBCiMnZ2EDHKeJxLhV2fwY5P4MtpxveQ7hONRYchA4xdEWvh0avq3mft2dqRJ0Z35sc9aczfdIwx3S9dR2QNeUUl3LdgJ472inem9a71pmZjugcyKNyPF5bH8966I5w8V8CbN/eycrSNh1SgWwBL50yujM8AsFprRaivC56tHYm9aEOV+LRsXJzs6eBrhYUHR9dD0iapPgshhDCfZ5DRJ/2XvXDzQmjbHba8B/Ovhv9FwJJ7YP9SKMyp+Vw24GBvx4xBHdh85AwJJ7Otfn6tNX9fspeDGTm8NaVXnSeAeLo48vKNPZgxsAM/7ztJbmHdK/qNlSTQzdycNYkMfWUNp86bP2NydXwGXQM9CPS0zigdpZRpQ5VzFzwfdyKbzm3dL+iLrrV1LxvbcUv1WQghhKXsHaDLOJi2GJ44Ajd9ApGj4dAvxhSPV8Lhs+th6/uQlVzz+axoct8QnB3s+MQGOx0u2HycpbEneHRUJ4ZGWm/3z2t7tqOwxBiB11xJAt3MLd2VSmpWPo98GWtWJTorr4jtx8/UafOUysQEe3LgZDZ5RcZPo1pr603gOLoBjv9mVJ8dLd9yXAghhPhDKw/oNgEmvgePJ8JtP8GAeyArCZY/Dm90h3nDjWQ674zNw/F2deL6XkF8uyuVrLwiq513V9JZnvshjhFdAnhgeITVzgvQp4M3/u7OLN/TfPdMkgS6GUs6ncehjPP0bu/FhkOneHfd4Rpfs/ZAJmUaRkbVfvvuysS096JMw75U4y2o1Kx8cgpKrLOAcN3L4NYWet9a93MJIYQQ5ewdoMNguOrf8NAOeHA7XPkclBYbyfT/OsGiaZDwI5RYL7m92K2DQykoLuPLbdapfp8+X8j9n++kjUcrXp8UY/WNUOztFGO7t2XNgYxm28YhCXQztirBWLX72qQYrukRyGu/HmTbsep/Wl6VkIGfmxM9rDyCpmewaUfC5LOA0b4BEFXXCvSxjXBsg7GyWqrPQgghbMkvEi57GO7bCPduhP53G3sPLJoKr3WBn/4GJ3YZC9utKCrQgwFhPny2+TilFq5rulhpmebhRbGczi1i7vQ+eLo4WinKC10dHUhhSRmrm2kbhyTQzdjqhAw6+rsS6ufKixOjCfFuzUMLd3Emt/KfkotLy1h3IIPhnQOs/tOor5szIT6t/1hIGJeWjVLQpW0dt/Be+xK4tTHmeAohhBD1pW00jHkBHk2AqV9B6FDY/hHMGwbvDIINr8KpRKtdbubgUFLO5rOqjiPt3lh5kI2Jp3h+fDebzmvuF+qDn5szy/c2zzYOSaCbqZyCYjYfOf1HK4Z7K0dmT+3NmdwiHl+8u9J+6O3HzpJdUGL19o1yPYO92G1aSBh3IpswP1dcnOowSfH4JqP6fNnDxhauQgghRH2zd4BOo2HSJ/D4QbjmdXB2h1XPwew+MGcgrP4PpO2pU2X6yq5taOfZivl1WEy4Kj6dt1cnMrlvCJP7ta/1ecxRsY2jfP1TcyJzoJupjYdOUVyqGVlhFF33IE+eviaKZ5bt54ONR7j78gvnPK9OSMfJ3o4hkXXffbAyMSFe/LAnjYycAuJPZv/R1nGJslL45FpI31f9CYsLwDUA+txm/WCFEEIIS7X2hr63G49zKUZvdPz3sOF/sP4V8OpgbCEedR0E9zNmUpvJwd6O6YM68MrPBziYnkOnNpa9g5t0Oo9HvoylWzsP/jW+m6V/slq5OjqQzzYfZ3VCBtf0aFcv16wvkkA3U6sSMvBo5UCfDt4XPD9jYAc2HznNKz8foG+oD73be1/wmgHhPrg52+afRUyIkTBvOHiK5DP53FzVT7/7vjGmakTfBC6+1Z+00xhwMm+3JCGEEKLeeAYb0zsG3GNsI35guZFMb3kPfp9tLH7vdBUE9zeSab9ONSbUN/drzxsrD/HJpmP85/pos0PJLyrlvs93APDutD613izFUv3D/mzjqE0C/eOeNJbFpnLfsI70au9d8wvqkSTQzVBpmWZNQgbDOgfgYH/hf0alFC9O7MHe1A08tHAXP84agpeLE0dP5XIkM5dbBnawWVzdgzyxt1N/rCKudAJHWSmsewUCusH18yz66VwIIYRolNz8oc+txqPgHBxcAfHLIG4Z7PzUOMbZA4J6G8l0UF8I7guuF74j7OPqxPie7ViyM5UnRncxawFgSWkZD32xk7i0bD68tS/tfeuv6GRvpxjTvQ1f70ghr6jEorbNguJSnv1+P5k5hayIS2dMt7Y8ProzEQFuNozYfJKdNEO7U7I4nVtU5Sxnz9aOzJ7Sm4ycAh5fvAet9R+LEmzV/wzQytGeLm3d2WqaBNKtsgkc+7+F04fgiickeRZCCNH8tPKEHjfB5AXwxDFjNN6EdyH6RmOu9IbX4IvJ8N+O8GZP+OZOY+b0yb1QVsqtg0PJLy5l8Y6aR9pprfnHsn2sjM/gueu6MaKL7b7HV+Xq6EAKistYk5Bp0esWbkkiM6eQD2/tyyOjOrHhUCaj31jPk9/s4eS5AhtFaz6pQDdDq+MzsLdTXNGp6l2FeoZ48eTYKJ7/IY6PfjvG6oQMOrVxI8THtj+ZxoR4sf9ENr6uTvi7X7TldlmpMdM5oKvRHyaEEEI0Z3Z2xmg8v0iImWo8V5QLJ2IhdTukbIOj62HvYuNzzh50D+7Ly37t+H3jIUr73YF9q6p7od9cdYgvtibz4PAIZgwKtf2fpxIDwnzxc3Ni+b40xvUINOs1BcWlvLvuMAPDfRgZ1YaRUW2YNrA9s1cn8vmW43y7K5XbLgvjvis62mwMX00kgW6GViVk0KeDN14uTtUed/tloWw+cpqXfopHa7hzaLjNY+sZ4sXnW5Lo2s4DpS4albf/Wzh1EG78WKrPQgghWiYnVwi9zHiAMbkj6zgkbYGk3yF5C5POr2EymrKXn4O23aH9IAi7AkKHGDspYlRw31h5iJv6BPPYVZ0a7I9jb6cY3a0tS3amkl9USmunmvuvPzdVn9+e0uuP5/zcnHn2um7cMSSM1349yHvrD7Nwy3HuHx7BzMGh9dbXXa7FZylaa77ffYKZH28lNSvf5tdLOp3HA5/vtNlcxNSsfOLTshllxlbcSin+e2MPAtxbUVKmzXpNXfUyLSS8ZAvvsjJY/1/w7wJdJ9g8DiGEEKJJUAq8Q6HnZLj2Dbj/d0r+epRHHJ7me/dJRu/0jk9g0RR4ORQ+vIrDX/2dpcsWM6qTNy9MjL60YFXPxkUHkl9cytoDNW+qUlBcytx1hxkU7svA8EsHCYT4uPD65Bh+fGgovTt489JPCVz1+nqyC4ptEXqVWnQF+tT5Qv6xdB8/7TsJwKwvdrHo7oE42lv/54qyMs2CLcd56acE8opK2XbsDCO6BFj9J6byHX/M7XPycnHivRl9WLIztV5WuHb0d+O+YR25oXfQhZ+IWwqZCXDjR1J9FkIIIarh6OpNx8ETeHjFQbpNv5wIHydI3gpH1pIb/yuhSe/wlZNGn3RFLRoCHYcbFWr/Lg3yPbZ/mA++rk78uDeNsdHVt3Es2HyczJxCZleoPlemazsP5t/Wn9UJ6dw+fzufbjrGgyMirRl2tVpsprJ8bxpXvb6eVfEZPDGmM29MjmHH8bO8uuKg1a+VfCaPaR9s4Zll++nTwZvXJ/ckI6eQRVuTrH6t1fHpdPB1oaO/q9mv6R7kyTPXdsXeyrsPVsbOTvG3MV2ICKjQs1VWZkze8Oss1Wchmhil1Bil1AGlVKJS6slKPt9eKbVGKbVLKbVHKXV1Q8QpRHMzpX97nBzs+GTTcXBwhrChJEY/wmVnnmaC2wJyxn+M6nEznE6En5+EdwfBi8Hw4VXw42NG1Tp1p7Gngo052NsxuntbVidkUFBcWuVx+UWlzF13hMEdfRlQSfW5MiO6tGFklwA+2HiU84X1t2FLi6tAn8kt4pll+/hhTxrRQZ68OqnnH8PItxw9w9x1hxkQ7sPwznVvZ9Bas3BrEi/8GG8aHxfNzf1CUErxxdZk3l13mJv7t7daFTqvqITfDp9m+oAODf52jUXiv4PMeLjhQ7Cr3x4mIUTtKaXsgTnAlUAKsE0p9Z3WOq7CYU8DX2mt31VKdQWWA6H1HqwQzYyvmzPX9mjHNztT+OuYzuQXlXLrR1txsLPjnTtG4O7jAr0mGgdnJcGxjZC229gRcfeXsO0D43N2DkYBq200BPYw+qjbRFu9Uj0uOpCFW5JYeyCDMd0rr0J/vuU4p84XMmdq9dXni80aGcn4Ob/x6e/HuH9YhBWirVmLSqB/3neSp5fu5Vx+MX8d3Zl7Lg+/YE7yP6/tyq6kszz21W6WzxpKW89Wtb5WalY+T36zhw2HTnFZhC8v39CDYO8/J1z8ZWQkUz/Ywlfbk7nFSitjf0s8TVFJWZXj6xqlsjJj8oZfJ+h2fUNHI4SwTH8gUWt9BEAptQgYD1RMoDVQvujBEzhRrxEK0YzNHBzKNztT+GjjUX7ed5Jz+cUsunvgpRO1vNobUz7KJ32UlUHWMSOZPrkXTu6BI2thzyLj864BEDEKIkZCxxHg4lPnWAeE+eDj6sSPe09WmkAb1efDFlWfy/UM8WJYZ38+2HCUWweF4mqjDeEqahEJ9NncIp79fj/LYk/QrZ0HC+4cQJe2l84gbuVoz5xpvbn27Y3M+mIXC+8acMlGJDXRWvPV9mSe/yGeMq3594TuTBvQ/pKK8KCOvvQL9eadNYeZ3C8EZ4e6V15XJ6Tj7uxAv9C6/0OvNwnfQ0YcTHxfqs9CND1BQMVhtCnAgIuOeRZYoZR6CHAFRtVPaEI0f9HBnvTp4M0bKw/haK/4eGZ/ugd51vxCOzvwCTce3Sq0TmanGYl04q9w8CfYvRCUHQT1MSXUV0K7mFp9v3awt2N0t7Ysi02loLj0knffjepzEe9Mq93EkFkjI5n4ziYWbD7OPVd0rNU5LNHse6DXHsjgqjfW8+OeNB4Z1YmlD1xWafJcrqO/G/+5vjtbj53hzVWHLLpW2rl8Zn68jb99s5fuQR788pfLmT6w8nYKpRQPj+zEyewCvtpW8zD0mpSVaVbFZ3B5J3+cHJrIX2t577NvBHS/oaGjEUJYrrJeMX3Rx1OA+VrrYOBq4DOlVKU3KaXU3Uqp7Uqp7ZmZlm26IERLdc/l4TjYKf53U0+GRPrV/ILqeARCzBRjQf9fD8Odq+DyJ4xRemtfgg9GwH8j4PNJsPJfsPdryIiHUvMmYIyLDiSvqJS1By78/51XVMLcdYe5LMKX/mG1KwL2bu/N0Eg/5q0/Qn5R1X3W1tLsK9BFJWX4uTkz/7Z+dGtnxk9lwPW9gtmUeJrZaxIZEOZb4z9IrTWLd6Tw/A9xlJRqnr22K7cMCsWuhkV5l0X40reDN++sPcykOlah95/IJiOnkBFdmlD7xoEfIX2factuqT4L0QSlACEVPg7m0haNO4AxAFrr35VSrQA/4JJ5VlrrecA8gL59+16ciAshKnFVt7bsfXa0WfOVLWJnb2wnHtwXhj9l7JJ4eDUkroK0WDi8CspMi/bsnYw+6jZdoU03COgG/p3AI+iC7+8Dw402juV70xjTve0fz3++OYlT54t4d1Td5lU/PDKSG+f+zudbjtt8b4tmn0Bf1a0tI6PaWDxh4l/juxGbnMVfvtzF8llDCfCovB86PbuAJ7/Zw5oDmfQP9eG/N/Wgg695EzCUUjw8KpIZH25l8fYUpg/sYFGMFa1KSEcpGNa56t0HGxWtjd5nn45SfRai6doGRCqlwoBU4GZg6kXHJAEjgflKqSigFSDlZSGsyOrJc2VcfIztxqNvND4uKTI2P8uIM4ph6XFwdAPs+fLP19g7gVcH8AkD7zAcfMKZFQJfxydRkN+ZVq1dyCsq4b31hxkS4VfnFtS+oT5cFuHL3HVHmD6wg003V2n2CTRQq/FsLk4OzJnWm+tmb+ThRbEsuHPABefRWvPtrlSe/W4/RaVlPHNNV2YOrrnqfLEhEX70bu/FO2sSmdQ3pNbtF6sTMugV4oWvm3PNBzcGB5YbCxcmzAX7FvHPUIhmR2tdopR6EPgFsAc+0lrvV0o9B2zXWn8HPAa8r5R6BKO9Y6bWWqrLQjR1Dk7GLohtuwOT/nw+74zR1nH6EJw5CmeOwNmjcPx3KMphJjDTDvTLj4FnMGdVGx4rcGdIQD/YfwK8w4yEu5V5XQMXmzUiksnzNrNwSxK3Dwmzxp+0UjbNXJRSY4A3MW6sH2itX7ro8/cCDwClwHng7ovGHzWoTm3ceW58d574eg+zVyfy8ChjQHdGdgF//3YvK+Mz6NvBm//e1JMwP/PnLldkVKE7cetHW1m8I5lpAyyvQqdnF7An5Rx/Hd25VjHUu/JeKp9wiL6poaMRQtSB1no5xmi6is89U+H3ccBl9R2XEKKBuPhcuBV5Oa0h9xQlpw/zzPwfGOqTzZWBeZzat5urnRLx3LkGdlY4vrWPqXIdCp4h4BUCnu1Nv4aAs1ullx8Q7svAcB/mrjvM1AHWGxV8MZsl0GbOB12otZ5rOv464DVMvXKNxU19gtl8+DRvrjpIvzBvMrIL+ed3+ykoLuXpcVHcdllYnTcguTzSj5gQL95Zc5ib+lhehV5j2n2wyYyvO/CTMTJn/DtSfRZCCCFaAqXAzR8HN39Ku7nw171p3NM1nFcLDvL1vYPoG+gIZ48ZVeuzR//8NXUHxH0HZRctVGztbUqs2xsP347gGwm+EcwaEcHUD7ZadVTwxWyZvdQ4H1RrnV3heFcuXb1dd0c3QMKPtX65Al52LeMytxMcnP+ZMZrO3Zmh0X54nV9nvHFZRwp42zuPX9PSObrgGzq3ca/xNRX5xafzimsRnXetp/JF8Y3MoRXGT5Q9Jjd0JEIIIYSoZ1f3COTL7cm8vvIgQyP96Fve+9w22nhcrKwMzp+ErGQ4l2xsDHMu2fj4dKKxsLEk/4/DBzm6stqtLYd/bUtJ3mAc/DsbCXZAV3Cs/R4fFdkygTZnPihKqQeARwEnYERlJ1JK3Q3cDdC+fXvLoshMgNiFlr3mIo7ABKXJty/F0UHhVGaHOmDdRDUYzSTHUvQxjU5zQJmZCGs0/QtKcHSwQ8U2kUkWdnZwzetSfRZCCCFaoMEdffFycSQrr5i/mNpjq2VnBx7tjMelqaSRYOekGX3XpxNRpxLxStpPp9QE7Db8DpQZx9212phpbQW2zGDMmQ+K1noOMEcpNRVjy9dbKzmm9qON+t9lPOrIHqi828Y6FLAtIYPb5m/jpaujubm/eT8orD2QwW0fb2P+1H4Ms8L240IIIYQQtuRob8ddQ8NJOZtHnw5W2PzNzg48g4xH+DAAvLXmznc3cebceVbMDMEp6zD4d6n7tcovabUzXcqc+aAVLQImVPP5Zm9YZ396BHsye00ixaVlZr1mdXwGLk72DLRw20shhBBCiIbywPAIXpzYw2bnLx/ScOxcCd8ku0HUteBUu4EPlbFlAv3HfFCllBPGfNDvKh6glKpYtx8HWLb1XzNj7E4YScrZfL7dmVrj8VprVsWnMyTCz6azDoUQQgghmprLI/3oGeLFHAsKk+ayWQKttS4ByueDxgNflc8HNU3cAHhQKbVfKRWL0Qd9SftGSzOiSwDRQeZVoRNO5nDiXEHTmb4hhBBCCFFPlFL8xYLCpCVsuorLjPmgD9vy+k1ReRX6zk+38+x3+6udL73j+FkAhkvvsxBCCCHEJSq2x17fOwhHe+vUjmUMQiM0MiqA3u29+HxLUo3HDu7oW+U240IIIYQQLZlSilkjIrlnwQ5ik7PqvF14OUmgGyGlFIvvHUxuUUmNx7o6yV+hEEIIIURVRkYFsO6vwwj2drHaOSX7aqTs7RQerRwbOgwhhBBCiCZNKWXV5BlsO4VDCCGEEEKIZkcSaCGEEEIIISwgCbQQQgghhBAWkARaCCGEEEIIC0gCLYQQQgghhAUkgRZCCCGEEMICkkALIYQQQghhAaW1bugYLKKUygSOV3jKDzjVQOHURVOMuynGDBJ3fWqKMUPt4u6gtfa3RTCi2dzrm2LMIHHXp6YYMzTNuGsbc6X3+iaXQF9MKbVda923oeOwVFOMuynGDBJ3fWqKMUPTjbslaYp/R00xZpC461NTjBmaZtzWjllaOIQQQgghhLCAJNBCCCGEEEJYoDkk0PMaOoBaaopxN8WYQeKuT00xZmi6cbckTfHvqCnGDBJ3fWqKMUPTjNuqMTf5HmghhBBCCCHqU3OoQAshhBBCCFFvmnQCrZQao5Q6oJRKVEo92dDxmEMpdUwptVcpFauU2t7Q8VRFKfWRUipDKbWvwnM+SqlflVKHTL96N2SMlaki7meVUqmmr3msUurqhozxYkqpEKXUGqVUvFJqv1LqYdPzjfrrXU3cjfbrrZRqpZTaqpTabYr5X6bnw5RSW0xf6y+VUk4NHaswNMX7PMi93paa4n0emua9vine56F+7vVNtoVDKWUPHASuBFKAbcAUrXVcgwZWA6XUMaCv1rpRz09USl0OnAc+1Vp3Nz33CnBGa/2S6RuZt9b6bw0Z58WqiPtZ4LzW+n8NGVtVlFKBQKDWeqdSyh3YAUwAZtKIv97VxD2JRvr1VkopwFVrfV4p5QhsBB4GHgWWaK0XKaXmAru11u82ZKyi6d7nQe71ttQU7/PQNO/1TfE+D/Vzr2/KFej+QKLW+ojWughYBIxv4JiaDa31euDMRU+PBz4x/f4TjP9EjUoVcTdqWus0rfVO0+9zgHggiEb+9a4m7kZLG86bPnQ0PTQwAvja9Hyj+1q3YHKft7GmeK9vivd5aJr3+qZ4n4f6udc35QQ6CEiu8HEKTeAvFeMvcIVSaodS6u6GDsZCbbTWaWD8pwICGjgeSzyolNpjeuuv0bw9djGlVCjQC9hCE/p6XxQ3NOKvt1LKXikVC2QAvwKHgSytdYnpkKZyL2kJmup9HuRe3xAa7X3nYk3xXt+U7vNg+2Pp2AwAAAQeSURBVHt9U06gVSXPNYV+lMu01r2BscADpreihG29C3QEYoA04NWGDadySik34BvgL1rr7IaOx1yVxN2ov95a61KtdQwQjFHhjKrssPqNSlShqd7nQe719a1R33cqaor3+qZ2nwfb3+ubcgKdAoRU+DgYONFAsZjt/9u7fxC5qjAM48/LbpQlIkEjIkgMYiohgoiIWAQRe1FJgkIQC0mjlSg2gmhhEyRoYzCFEJWARlOJElQURS2Mf4KdLBbGTVIEESRI8lnMGRh0d91BZ+be3ecHw5w5c3c4c5Z5+ebOufdW1S/t/gxwjME/tS+W2nqo4bqoMzMez5pU1VL7IF0CDtHBOW9rtN4GjlTVO6278/O93Lj7MN8AVXUe+Bi4A9iSZL491Yss2SB6mfNg1k9bX3Knj1nf55yHyWV9nwvor4Ed7YjKy4A9wPEZj2lVSTa3Rfgk2QzcC/yw+l91ynFgX2vvA96b4VjWbBhMzX10bM7bwQ6vAT9W1YGRpzo93yuNu8vzneSaJFtaewG4h8Gavo+AB9pmnZvrDax3OQ9m/Sx0OXeG+pj1fcx5mE7W9/YsHADttCkvAXPA4ap6YcZDWlWSGxnsiQCYB97o6piTvAnsArYCS8CzwLvAUWAb8DPwYFV16kCOFca9i8HPTAUsAo8N15t1QZK7gE+B74FLrfsZBuvMOjvfq4x7Lx2d7yQ7GRw4MsdgB8LRqnqufTbfAq4CvgEerqoLsxuphvqW82DWT1ofcx76mfV9zHmYTtb3uoCWJEmSpq3PSzgkSZKkqbOAliRJksZgAS1JkiSNwQJakiRJGoMFtCRJkjQGC2j1VpKLSU6O3J7+H197e5JOnddSkjYis15dNP/vm0id9Ue7TKckaf0y69U57oHWupNkMcmLSb5qt5ta/w1JTiT5rt1va/3XJjmW5Nt2u7O91FySQ0lOJfmgXc1IktQBZr1myQJafbbwt5/1do8891tV3Q68zOAqZrT261W1EzgCHGz9B4FPquoW4FbgVOvfAbxSVTcD54H7J/x+JEn/ZNarc7wSoXorye9VdcUy/YvA3VX1U5JNwK9VdXWSc8B1VfVn6z9dVVuTnAWuH72cZ5LtwIdVtaM9fgrYVFXPT/6dSZKGzHp1kXugtV7VCu2VtlnOhZH2RTxmQJK6xqzXTFhAa73aPXL/RWt/Duxp7YeAz1r7BLAfIMlckiunNUhJ0n9i1msm/JalPltIcnLk8ftVNTy90eVJvmTwJXFv63scOJzkSeAs8EjrfwJ4NcmjDPY+7AdOT3z0kqS1MOvVOa6B1rrT1sXdVlXnZj0WSdJkmPWaJZdwSJIkSWNwD7QkSZI0BvdAS5IkSWOwgJYkSZLGYAEtSZIkjcECWpIkSRqDBbQkSZI0BgtoSZIkaQx/ARR+nSxzt/58AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "f, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n", + "t = f.suptitle('Basic CNN Performance', fontsize=12)\n", + "f.subplots_adjust(top=0.85, wspace=0.3)\n", + "\n", + "epoch_list = list(range(1,31))\n", + "ax1.plot(epoch_list, history.history['acc'], label='Train Accuracy')\n", + "ax1.plot(epoch_list, history.history['val_acc'], label='Validation Accuracy')\n", + "ax1.set_xticks(np.arange(0, 31, 5))\n", + "ax1.set_ylabel('Accuracy Value')\n", + "ax1.set_xlabel('Epoch')\n", + "ax1.set_title('Accuracy')\n", + "l1 = ax1.legend(loc=\"best\")\n", + "\n", + "ax2.plot(epoch_list, history.history['loss'], label='Train Loss')\n", + "ax2.plot(epoch_list, history.history['val_loss'], label='Validation Loss')\n", + "ax2.set_xticks(np.arange(0, 31, 5))\n", + "ax2.set_ylabel('Loss Value')\n", + "ax2.set_xlabel('Epoch')\n", + "ax2.set_title('Loss')\n", + "l2 = ax2.legend(loc=\"best\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Model: \"sequential_3\"\n", + "_________________________________________________________________\n", + "Layer (type) Output Shape Param # \n", + "=================================================================\n", + "dense_5 (Dense) (None, 256) 1179904 \n", + "_________________________________________________________________\n", + "dropout_2 (Dropout) (None, 256) 0 \n", + "_________________________________________________________________\n", + "dense_6 (Dense) (None, 3) 771 \n", + "=================================================================\n", + "Total params: 1,180,675\n", + "Trainable params: 1,180,675\n", + "Non-trainable params: 0\n", + "_________________________________________________________________\n" + ] + } + ], + "source": [ + "import keras as ks\n", + "model = ks.models.load_model('train3.h5')\n", + "model.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [], + "source": [ + "#訓練結果提取，建立混淆矩陣\n", + "import pandas as pd\n", + "import tensorflow as tf\n", + "import sklearn\n", + "from sklearn.metrics import confusion_matrix\n", + "predictions = model.predict_classes(test_features)" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[1, 0, 0, 1, 0, 2, 1, 2, 2, 1, 0, 2, 0, 0, 0]\n", + "(15,)\n" + ] + } + ], + "source": [ + "from numpy import argmax\n", + "from keras.utils.np_utils import to_categorical\n", + "test_labels_change = [0]*15\n", + "for i in range(12):\n", + " if(np.array_equal(test_labels[i],[0,0,1])):\n", + " test_labels_change[i] = 2\n", + " elif(np.array_equal(test_labels[i],[0,1,0])):\n", + " test_labels_change[i] = 1\n", + " elif(np.array_equal(test_labels[i],[1,0,0])):\n", + " test_labels_change[i] = 0\n", + "\n", + "print(test_labels_change)\n", + "test_labels_change = np.asarray(test_labels_change)\n", + "print(test_labels_change.shape)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(15,)\n", + "(15,)\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
預測值012
實際值
0511
1040
2004
\n", + "
" + ], + "text/plain": [ + "預測值 0 1 2\n", + "實際值 \n", + "0 5 1 1\n", + "1 0 4 0\n", + "2 0 0 4" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "print(test_labels_change.shape)\n", + "print(predictions.shape)\n", + "pd.crosstab(test_labels_change, predictions, rownames=['實際值'], colnames=['預測值'])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.5" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/Facial-Keypoint-Detection/Readme.md b/Facial-Keypoint-Detection/Readme.md index 62d7a235..c7ab91f8 100644 --- a/Facial-Keypoint-Detection/Readme.md +++ b/Facial-Keypoint-Detection/Readme.md @@ -1,43 +1,43 @@ -# Objective -Here we convert the https://www.kaggle.com/competitions/facial-keypoints-detection code to kfp-pipeline -The objective of this task is to predict keypoint positions on face images - -# Testing enviornment -The pipeline is tested on `Kubeflow 1.4` and `kfp 1.1.2` , it should be compatible with previous releases of Kubeflow . kfp version used for testing is 1.1.2 which can be installed as `pip install kfp==1.1.2` - -# Components used - -## Docker -Docker is used to create an enviornment to run each component. - -## Kubeflow pipelines -Kubeflow pipelines connect each docker component and create a pipeline. Each Kubeflow pipeline is reproducable workflow wherein we pass input arguments and run entire workflow. - -# Docker -We start with creating a docker account on dockerhub (https://hub.docker.com/). We signup with our individual email. After signup is compelete login to docker using your username and password using the command `docker login` on your terminal - -## Build train image -Navigate to `train` directory, create a folder named `my_data` and put your `training.zip` and `test.zip` data from Kaggle repo in this folder and build docker image using : -``` -docker build -t /: . -``` -In my case this is: -``` -docker build -t hubdocker76/demotrain:v1 . -``` - -## Build evaluate image -Navigate to eval directory and build docker image using : -``` -docker build -t /: . -``` -In my case this is: -``` -docker build -t hubdocker76/demoeval:v2 . -``` -# Kubeflow pipelines - -Go to generate-pipeline and run `python3 my_pipeline.py` this will generate a yaml file. which we can upload to Kubeflow pipelines UI and create a Run from it. - -# Sample pipeline to run on Kubeflow -Navigate to directory `geneate-pipeline` and run `python3 my_pipeline.py` this will generate yaml file. I have named this yaml as `face_pipeline_01.yaml`. Please upload this pipeline on Kubeflow and start a Run. +# Objective +Here we convert the https://www.kaggle.com/competitions/facial-keypoints-detection code to kfp-pipeline +The objective of this task is to predict keypoint positions on face images + +# Testing enviornment +The pipeline is tested on `Kubeflow 1.4` and `kfp 1.1.2` , it should be compatible with previous releases of Kubeflow . kfp version used for testing is 1.1.2 which can be installed as `pip install kfp==1.1.2` + +# Components used + +## Docker +Docker is used to create an enviornment to run each component. + +## Kubeflow pipelines +Kubeflow pipelines connect each docker component and create a pipeline. Each Kubeflow pipeline is reproducable workflow wherein we pass input arguments and run entire workflow. + +# Docker +We start with creating a docker account on dockerhub (https://hub.docker.com/). We signup with our individual email. After signup is compelete login to docker using your username and password using the command `docker login` on your terminal + +## Build train image +Navigate to `train` directory, create a folder named `my_data` and put your `training.zip` and `test.zip` data from Kaggle repo in this folder and build docker image using : +``` +docker build -t /: . +``` +In my case this is: +``` +docker build -t hubdocker76/demotrain:v1 . +``` + +## Build evaluate image +Navigate to eval directory and build docker image using : +``` +docker build -t /: . +``` +In my case this is: +``` +docker build -t hubdocker76/demoeval:v2 . +``` +# Kubeflow pipelines + +Go to generate-pipeline and run `python3 my_pipeline.py` this will generate a yaml file. which we can upload to Kubeflow pipelines UI and create a Run from it. + +# Sample pipeline to run on Kubeflow +Navigate to directory `geneate-pipeline` and run `python3 my_pipeline.py` this will generate yaml file. I have named this yaml as `face_pipeline_01.yaml`. Please upload this pipeline on Kubeflow and start a Run. diff --git a/Facial-Keypoint-Detection/generate-pipeline/my_pipeline.py b/Facial-Keypoint-Detection/generate-pipeline/my_pipeline.py index 9ec0ee6e..39ca3e21 100644 --- a/Facial-Keypoint-Detection/generate-pipeline/my_pipeline.py +++ b/Facial-Keypoint-Detection/generate-pipeline/my_pipeline.py @@ -1,42 +1,42 @@ -import kfp -from kfp import dsl - -def SendMsg(trial, epoch, patience): - vop = dsl.VolumeOp(name="pvc", - resource_name="pvc", size='1Gi', - modes=dsl.VOLUME_MODE_RWO) - - return dsl.ContainerOp( - name = 'Train', - image = 'hubdocker76/demotrain:v1', - command = ['python3', 'train.py'], - arguments=[ - '--trial', trial, - '--epoch', epoch, - '--patience', patience - ], - pvolumes={ - '/data': vop.volume - } - ) - -def GetMsg(comp1): - return dsl.ContainerOp( - name = 'Evaluate', - image = 'hubdocker76/demoeval:v2', - pvolumes={ - '/data': comp1.pvolumes['/data'] - }, - command = ['python3', 'eval.py'] - ) - -@dsl.pipeline( - name = 'face pipeline', - description = 'pipeline to detect facial landmarks') -def passing_parameter(trial, epoch, patience): - comp1 = SendMsg(trial, epoch, patience) - comp2 = GetMsg(comp1) - -if __name__ == '__main__': - import kfp.compiler as compiler - compiler.Compiler().compile(passing_parameter, __file__ + '.yaml') +import kfp +from kfp import dsl + +def SendMsg(trial, epoch, patience): + vop = dsl.VolumeOp(name="pvc", + resource_name="pvc", size='1Gi', + modes=dsl.VOLUME_MODE_RWO) + + return dsl.ContainerOp( + name = 'Train', + image = 'hubdocker76/demotrain:v1', + command = ['python3', 'train.py'], + arguments=[ + '--trial', trial, + '--epoch', epoch, + '--patience', patience + ], + pvolumes={ + '/data': vop.volume + } + ) + +def GetMsg(comp1): + return dsl.ContainerOp( + name = 'Evaluate', + image = 'hubdocker76/demoeval:v2', + pvolumes={ + '/data': comp1.pvolumes['/data'] + }, + command = ['python3', 'eval.py'] + ) + +@dsl.pipeline( + name = 'face pipeline', + description = 'pipeline to detect facial landmarks') +def passing_parameter(trial, epoch, patience): + comp1 = SendMsg(trial, epoch, patience) + comp2 = GetMsg(comp1) + +if __name__ == '__main__': + import kfp.compiler as compiler + compiler.Compiler().compile(passing_parameter, __file__ + '.yaml') diff --git a/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kale.ipynb b/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kale.ipynb index ed074e81..cfb4a1cc 100644 --- a/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kale.ipynb +++ b/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kale.ipynb @@ -1,1024 +1,1024 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# 🪙 G-Research Crypto Kale Pipeline\n", - "![](./images/vector-blockchain-poster.jpg)\n", - "\n", - "---\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to forecast short term returns in 14 popular cryptocurrencies. The dataset provided contains information on historic trades for several cryptoassets, such as Bitcoin and Ethereum. \n", - "\n", - "> G-Research is a leading quantitative research and technology company. By using the latest scientific techniques, they produce world-beating predictive research and build advanced technology to analyse the world's data." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Install necessary packages\n", - "\n", - "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", - "\n", - "NOTE: Do not forget to use the --user argument. It is necessary if you want to use Kale to transform this notebook into a Kubeflow pipeline. After installing python packages, restart notebook kernel before proceeding." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "!pip install -r requirements.txt --user --quiet" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Imports\n", - "\n", - "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "tags": [ - "imports" - ] - }, - "outputs": [], - "source": [ - "import os, random, subprocess\n", - "import pandas as pd\n", - "import numpy as np\n", - "import time, datetime, zipfile\n", - "import joblib, talib\n", - "from tqdm import tqdm\n", - "import lightgbm as lgb\n", - "\n", - "import warnings\n", - "warnings.filterwarnings(\"ignore\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Project hyper-parameters\n", - "\n", - "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "tags": [ - "pipeline-parameters" - ] - }, - "outputs": [], - "source": [ - "# Hyper-parameters\n", - "LR = 0.01\n", - "N_EST = 1200" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "Set random seed for reproducibility" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "def fix_all_seeds(seed):\n", - " np.random.seed(seed)\n", - " random.seed(seed)\n", - " os.environ['PYTHONHASHSEED'] = str(seed)\n", - "\n", - "fix_all_seeds(2022)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Download data\n", - "\n", - "In this section, we download the data from kaggle using the Kaggle API credentials" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "tags": [ - "block:download_data" - ] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "CompletedProcess(args=['kaggle', 'competitions', 'download', '-c', 'g-research-crypto-forecasting'], returncode=0)" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# setup kaggle environment for data download\n", - "dataset = \"g-research-crypto-forecasting\"\n", - "\n", - "# setup kaggle environment for data download\n", - "with open('/secret/kaggle-secret/password', 'r') as file:\n", - " kaggle_key = file.read().rstrip()\n", - "with open('/secret/kaggle-secret/username', 'r') as file:\n", - " kaggle_user = file.read().rstrip()\n", - "\n", - "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", - "\n", - "# download kaggle's g-research-crypto-forecast data\n", - "subprocess.run([\"kaggle\",\"competitions\", \"download\", \"-c\", dataset])" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "tags": [ - "block:" - ] - }, - "outputs": [], - "source": [ - "# path to download to\n", - "data_path = 'data'\n", - "\n", - "# extract g-research-crypto-forecasting.zip to load_data_path\n", - "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", - " zip_ref.extractall(data_path, members=['train.csv', 'asset_details.csv'])" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Load the dataset\n", - "\n", - "First, let us load and analyze the data.\n", - "\n", - "The data is in csv format, thus, we use the handy read_csv pandas method." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "tags": [ - "block:load_data", - "prev:download_data" - ] - }, - "outputs": [], - "source": [ - "TRAIN_CSV = f'{data_path}/train.csv'\n", - "ASSET_DETAILS_CSV = f'{data_path}/asset_details.csv'" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "df_train = pd.read_csv(TRAIN_CSV)" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "(24236806, 10)" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Asset_IDWeightAsset_Name
104.304065Binance Coin
216.779922Bitcoin
022.397895Bitcoin Cash
1034.406719Cardano
1343.555348Dogecoin
351.386294EOS.IO
565.894403Ethereum
472.079442Ethereum Classic
1181.098612IOTA
692.397895Litecoin
12101.098612Maker
7111.609438Monero
9122.079442Stellar
8131.791759TRON
\n", - "
" - ], - "text/plain": [ - " Asset_ID Weight Asset_Name\n", - "1 0 4.304065 Binance Coin\n", - "2 1 6.779922 Bitcoin\n", - "0 2 2.397895 Bitcoin Cash\n", - "10 3 4.406719 Cardano\n", - "13 4 3.555348 Dogecoin\n", - "3 5 1.386294 EOS.IO\n", - "5 6 5.894403 Ethereum\n", - "4 7 2.079442 Ethereum Classic\n", - "11 8 1.098612 IOTA\n", - "6 9 2.397895 Litecoin\n", - "12 10 1.098612 Maker\n", - "7 11 1.609438 Monero\n", - "9 12 2.079442 Stellar\n", - "8 13 1.791759 TRON" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_asset_details = pd.read_csv(ASSET_DETAILS_CSV).sort_values(\"Asset_ID\")\n", - "df_asset_details" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "df_train['datetime'] = pd.to_datetime(df_train['timestamp'], unit='s')\n", - "df_train = df_train[df_train['datetime'] >= '2020-01-01 00:00:00'].copy()" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "(12228898, 11)" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "Timestamp('2021-09-21 00:00:00')" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train['datetime'].max()" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "timestamp 0\n", - "Asset_ID 0\n", - "Count 0\n", - "Open 0\n", - "High 0\n", - "Low 0\n", - "Close 0\n", - "Volume 0\n", - "VWAP 9\n", - "Target 262453\n", - "datetime 0\n", - "dtype: int64" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.isna().sum()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "### Define Pipeline Functions" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "tags": [ - "functions" - ] - }, - "outputs": [], - "source": [ - "# define the evaluation metric\n", - "def weighted_correlation(a, train_data):\n", - " \n", - " weights = train_data.add_w.values.flatten()\n", - " b = train_data.get_label()\n", - " \n", - " \n", - " w = np.ravel(weights)\n", - " a = np.ravel(a)\n", - " b = np.ravel(b)\n", - "\n", - " sum_w = np.sum(w)\n", - " mean_a = np.sum(a * w) / sum_w\n", - " mean_b = np.sum(b * w) / sum_w\n", - " var_a = np.sum(w * np.square(a - mean_a)) / sum_w\n", - " var_b = np.sum(w * np.square(b - mean_b)) / sum_w\n", - "\n", - " cov = np.sum((a * b * w)) / np.sum(w) - mean_a * mean_b\n", - " corr = cov / np.sqrt(var_a * var_b)\n", - "\n", - " return 'eval_wcorr', corr, True" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "tags": [ - "functions" - ] - }, - "outputs": [], - "source": [ - "def RSI(df, n):\n", - " return talib.RSI(df['Close'], n)\n", - "\n", - "def ATR(df, n):\n", - " return talib.ATR(df[\"High\"], df.Low, df.Close, n)\n", - "\n", - "#Create a function to calculate the Double Exponential Moving Average (DEMA)\n", - "def DEMA(data, time_period):\n", - " #Calculate the Exponential Moving Average for some time_period (in days)\n", - " EMA = data['Close'].ewm(span=time_period, adjust=False).mean()\n", - " #Calculate the DEMA\n", - " DEMA = 2*EMA - EMA.ewm(span=time_period, adjust=False).mean()\n", - " return DEMA\n", - "\n", - "def upper_shadow(df):\n", - " return df['High'] - np.maximum(df['Close'], df['Open'])\n", - "\n", - "def lower_shadow(df):\n", - " return np.minimum(df['Close'], df['Open']) - df['Low']" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Feature Engineering" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "tags": [ - "block:feature_engineering", - "prev:load_data" - ] - }, - "outputs": [], - "source": [ - "def get_features(df, \n", - " asset_id, \n", - " train=True):\n", - " '''\n", - " This function takes a dataframe with all asset data and return the lagged features for a single asset.\n", - " \n", - " df - Full dataframe with all assets included\n", - " asset_id - integer from 0-13 inclusive to represent a cryptocurrency asset\n", - " train - True - you are training your model\n", - " - False - you are submitting your model via api\n", - " '''\n", - " # filter based on asset id\n", - " df = df[df['Asset_ID']==asset_id]\n", - " \n", - " # sort based on time stamp\n", - " df = df.sort_values('timestamp')\n", - " \n", - " if train == True:\n", - " df_feat = df.copy()\n", - " \n", - " # define a train_flg column to split your data into train and validation\n", - " totimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n", - " valid_window = [totimestamp(\"01/05/2021\")]\n", - " \n", - " df_feat['train_flg'] = np.where(df_feat['timestamp']>=valid_window[0], 0,1)\n", - " df_feat = df_feat[['timestamp','Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','Target','train_flg']].copy()\n", - " else:\n", - " df = df.sort_values('row_id')\n", - " df_feat = df[['Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','row_id']].copy()\n", - " \n", - " for i in tqdm([30, 120, 240]):\n", - " # creating technical indicators\n", - " df_feat[f'RSI_{i}'] = RSI(df_feat, i)\n", - " df_feat[f'ATR_{i}'] = ATR(df_feat, i)\n", - " df_feat[f'DEMA_{i}'] = DEMA(df_feat, i)\n", - "\n", - " for i in tqdm([30, 120, 240]):\n", - " # creating lag features\n", - " df_feat[f'sma_{i}'] = df_feat['Close'].rolling(i).mean()/df_feat['Close'] -1\n", - " df_feat[f'return_{i}'] = df_feat['Close']/df_feat['Close'].shift(i) -1\n", - " \n", - " # new featu# creating technical indicators featureses\n", - " df_feat['HL'] = np.log(df_feat['High'] - df_feat['Low'])\n", - " df_feat['OC'] = np.log(df_feat['Close'] - df_feat['Open'])\n", - " \n", - " df_feat['lower_shadow'] = np.log(lower_shadow(df)) \n", - " df_feat['upper_shadow'] = np.log(upper_shadow(df))\n", - " \n", - " # replace inf with nan\n", - " df_feat.replace([np.inf, -np.inf], np.nan, inplace=True)\n", - " \n", - " # datetime features\n", - " df_feat['Date'] = pd.to_datetime(df_feat['timestamp'], unit='s')\n", - " df_feat['Day'] = df_feat['Date'].dt.weekday.astype(np.int32)\n", - " df_feat[\"dayofyear\"] = df_feat['Date'].dt.dayofyear\n", - " df_feat[\"weekofyear\"] = df_feat['Date'].dt.weekofyear\n", - " df_feat[\"season\"] = ((df_feat['Date'].dt.month)%12 + 3)//3\n", - " \n", - "\n", - " df_feat = df_feat.drop(['Open','Close','High','Low', 'Volume', 'Date'], axis=1)\n", - " \n", - " # fill nan values with 0\n", - " df_feat = df_feat.fillna(0)\n", - " \n", - " return df_feat" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 7.64it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 12.66it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 7.61it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 16.32it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "2\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 9.89it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 22.05it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "3\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 8.52it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 17.92it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "4\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 10.35it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 22.56it/s]\n" - ] - } - ], - "source": [ - "# create your feature dataframe for each asset and concatenate\n", - "feature_df = pd.DataFrame()\n", - "for i in range(14):\n", - " print(i)\n", - " feature_df = pd.concat([feature_df,get_features(df_train,i,train=True)])" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Merge Assets Features" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:merge_assets_features", - "prev:load_data", - "prev:feature_engineering" - ] - }, - "outputs": [], - "source": [ - "# assign weight column feature dataframe\n", - "feature_df = pd.merge(feature_df, df_asset_details[['Asset_ID','Weight']], how='left', on=['Asset_ID'])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "feature_df.columns" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Modelling" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:modelling", - "prev:merge_assets_features" - ] - }, - "outputs": [], - "source": [ - "# define features for LGBM\n", - "features = ['Asset_ID', 'RSI_30', 'ATR_30',\n", - " 'DEMA_30', 'RSI_120', 'ATR_120', 'DEMA_120', 'RSI_240', 'ATR_240',\n", - " 'DEMA_240', 'sma_30', 'return_30', 'sma_120', 'return_120', 'sma_240',\n", - " 'return_240', 'HL', 'OC', 'lower_shadow', 'upper_shadow', 'Day',\n", - " 'dayofyear', 'weekofyear', 'season']\n", - "categoricals = ['Asset_ID']" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# define train and validation weights and datasets\n", - "weights_train = feature_df.query('train_flg == 1')[['Weight']]\n", - "weights_test = feature_df.query('train_flg == 0')[['Weight']]\n", - "\n", - "train_dataset = lgb.Dataset(feature_df.query('train_flg == 1')[features], \n", - " feature_df.query('train_flg == 1')['Target'].values, \n", - " feature_name = features,\n", - " categorical_feature= categoricals)\n", - "val_dataset = lgb.Dataset(feature_df.query('train_flg == 0')[features], \n", - " feature_df.query('train_flg == 0')['Target'].values, \n", - " feature_name = features,\n", - " categorical_feature= categoricals)\n", - "\n", - "train_dataset.add_w = weights_train\n", - "val_dataset.add_w = weights_test\n", - "\n", - "evals_result = {}\n", - "params = {'n_estimators': int(N_EST),\n", - " 'objective': 'regression',\n", - " 'metric': 'rmse',\n", - " 'boosting_type': 'gbdt',\n", - " 'max_depth': -1, \n", - " 'learning_rate': float(LR),\n", - " 'seed': 2022,\n", - " 'verbose': -1,\n", - " }\n", - "\n", - "# train LGBM2\n", - "model = lgb.train(params = params,\n", - " train_set = train_dataset, \n", - " valid_sets = [val_dataset],\n", - " early_stopping_rounds=60,\n", - " verbose_eval = 30,\n", - " feval=weighted_correlation,\n", - " evals_result = evals_result \n", - " )\n", - "\n", - "joblib.dump(model, 'lgb.jl')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "fea_imp = pd.DataFrame({'imp':model.feature_importance(), 'col': features})\n", - "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", - "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Evaluation" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:evaluation_result", - "prev:modelling" - ] - }, - "outputs": [], - "source": [ - "model = joblib.load('lgb.jl')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "root_mean_squared_error = model.best_score.get('valid_0').get('rmse')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "weighted_correlation = model.best_score.get('valid_0').get('eval_wcorr')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Pipeline Metrics" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [], - "source": [ - "print(root_mean_squared_error)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [], - "source": [ - "print(weighted_correlation)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "kubeflow_notebook": { - "autosnapshot": true, - "experiment": { - "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", - "name": "g-research-crypto-forecasting" - }, - "experiment_name": "g-research-crypto-forecasting", - "katib_metadata": { - "algorithm": { - "algorithmName": "grid" - }, - "maxFailedTrialCount": 3, - "maxTrialCount": 12, - "objective": { - "objectiveMetricName": "", - "type": "minimize" - }, - "parallelTrialCount": 3, - "parameters": [] - }, - "katib_run": false, - "pipeline_description": "forecasting short term returns in 14 popular cryptocurrencies.", - "pipeline_name": "g-research-crypto-forecasting-pipeline", - "snapshot_volumes": true, - "steps_defaults": [ - "label:access-ml-pipeline:true", - "label:kaggle-secret:true", - "label:access-rok:true" - ], - "volume_access_mode": "rwm", - "volumes": [ - { - "annotations": [], - "mount_point": "/home/jovyan", - "name": "demo-workspace-fb99v", - "size": 15, - "size_type": "Gi", - "snapshot": false, - "type": "clone" - } - ] - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# 🪙 G-Research Crypto Kale Pipeline\n", + "![](./images/vector-blockchain-poster.jpg)\n", + "\n", + "---\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to forecast short term returns in 14 popular cryptocurrencies. The dataset provided contains information on historic trades for several cryptoassets, such as Bitcoin and Ethereum. \n", + "\n", + "> G-Research is a leading quantitative research and technology company. By using the latest scientific techniques, they produce world-beating predictive research and build advanced technology to analyse the world's data." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Install necessary packages\n", + "\n", + "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", + "\n", + "NOTE: Do not forget to use the --user argument. It is necessary if you want to use Kale to transform this notebook into a Kubeflow pipeline. After installing python packages, restart notebook kernel before proceeding." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "!pip install -r requirements.txt --user --quiet" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Imports\n", + "\n", + "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "imports" + ] + }, + "outputs": [], + "source": [ + "import os, random, subprocess\n", + "import pandas as pd\n", + "import numpy as np\n", + "import time, datetime, zipfile\n", + "import joblib, talib\n", + "from tqdm import tqdm\n", + "import lightgbm as lgb\n", + "\n", + "import warnings\n", + "warnings.filterwarnings(\"ignore\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Project hyper-parameters\n", + "\n", + "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "pipeline-parameters" + ] + }, + "outputs": [], + "source": [ + "# Hyper-parameters\n", + "LR = 0.01\n", + "N_EST = 1200" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "Set random seed for reproducibility" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "def fix_all_seeds(seed):\n", + " np.random.seed(seed)\n", + " random.seed(seed)\n", + " os.environ['PYTHONHASHSEED'] = str(seed)\n", + "\n", + "fix_all_seeds(2022)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Download data\n", + "\n", + "In this section, we download the data from kaggle using the Kaggle API credentials" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "tags": [ + "block:download_data" + ] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "CompletedProcess(args=['kaggle', 'competitions', 'download', '-c', 'g-research-crypto-forecasting'], returncode=0)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# setup kaggle environment for data download\n", + "dataset = \"g-research-crypto-forecasting\"\n", + "\n", + "# setup kaggle environment for data download\n", + "with open('/secret/kaggle-secret/password', 'r') as file:\n", + " kaggle_key = file.read().rstrip()\n", + "with open('/secret/kaggle-secret/username', 'r') as file:\n", + " kaggle_user = file.read().rstrip()\n", + "\n", + "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", + "\n", + "# download kaggle's g-research-crypto-forecast data\n", + "subprocess.run([\"kaggle\",\"competitions\", \"download\", \"-c\", dataset])" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "tags": [ + "block:" + ] + }, + "outputs": [], + "source": [ + "# path to download to\n", + "data_path = 'data'\n", + "\n", + "# extract g-research-crypto-forecasting.zip to load_data_path\n", + "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", + " zip_ref.extractall(data_path, members=['train.csv', 'asset_details.csv'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Load the dataset\n", + "\n", + "First, let us load and analyze the data.\n", + "\n", + "The data is in csv format, thus, we use the handy read_csv pandas method." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "tags": [ + "block:load_data", + "prev:download_data" + ] + }, + "outputs": [], + "source": [ + "TRAIN_CSV = f'{data_path}/train.csv'\n", + "ASSET_DETAILS_CSV = f'{data_path}/asset_details.csv'" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "df_train = pd.read_csv(TRAIN_CSV)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(24236806, 10)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Asset_IDWeightAsset_Name
104.304065Binance Coin
216.779922Bitcoin
022.397895Bitcoin Cash
1034.406719Cardano
1343.555348Dogecoin
351.386294EOS.IO
565.894403Ethereum
472.079442Ethereum Classic
1181.098612IOTA
692.397895Litecoin
12101.098612Maker
7111.609438Monero
9122.079442Stellar
8131.791759TRON
\n", + "
" + ], + "text/plain": [ + " Asset_ID Weight Asset_Name\n", + "1 0 4.304065 Binance Coin\n", + "2 1 6.779922 Bitcoin\n", + "0 2 2.397895 Bitcoin Cash\n", + "10 3 4.406719 Cardano\n", + "13 4 3.555348 Dogecoin\n", + "3 5 1.386294 EOS.IO\n", + "5 6 5.894403 Ethereum\n", + "4 7 2.079442 Ethereum Classic\n", + "11 8 1.098612 IOTA\n", + "6 9 2.397895 Litecoin\n", + "12 10 1.098612 Maker\n", + "7 11 1.609438 Monero\n", + "9 12 2.079442 Stellar\n", + "8 13 1.791759 TRON" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_asset_details = pd.read_csv(ASSET_DETAILS_CSV).sort_values(\"Asset_ID\")\n", + "df_asset_details" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "df_train['datetime'] = pd.to_datetime(df_train['timestamp'], unit='s')\n", + "df_train = df_train[df_train['datetime'] >= '2020-01-01 00:00:00'].copy()" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(12228898, 11)" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Timestamp('2021-09-21 00:00:00')" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train['datetime'].max()" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "timestamp 0\n", + "Asset_ID 0\n", + "Count 0\n", + "Open 0\n", + "High 0\n", + "Low 0\n", + "Close 0\n", + "Volume 0\n", + "VWAP 9\n", + "Target 262453\n", + "datetime 0\n", + "dtype: int64" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.isna().sum()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "### Define Pipeline Functions" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "tags": [ + "functions" + ] + }, + "outputs": [], + "source": [ + "# define the evaluation metric\n", + "def weighted_correlation(a, train_data):\n", + " \n", + " weights = train_data.add_w.values.flatten()\n", + " b = train_data.get_label()\n", + " \n", + " \n", + " w = np.ravel(weights)\n", + " a = np.ravel(a)\n", + " b = np.ravel(b)\n", + "\n", + " sum_w = np.sum(w)\n", + " mean_a = np.sum(a * w) / sum_w\n", + " mean_b = np.sum(b * w) / sum_w\n", + " var_a = np.sum(w * np.square(a - mean_a)) / sum_w\n", + " var_b = np.sum(w * np.square(b - mean_b)) / sum_w\n", + "\n", + " cov = np.sum((a * b * w)) / np.sum(w) - mean_a * mean_b\n", + " corr = cov / np.sqrt(var_a * var_b)\n", + "\n", + " return 'eval_wcorr', corr, True" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "tags": [ + "functions" + ] + }, + "outputs": [], + "source": [ + "def RSI(df, n):\n", + " return talib.RSI(df['Close'], n)\n", + "\n", + "def ATR(df, n):\n", + " return talib.ATR(df[\"High\"], df.Low, df.Close, n)\n", + "\n", + "#Create a function to calculate the Double Exponential Moving Average (DEMA)\n", + "def DEMA(data, time_period):\n", + " #Calculate the Exponential Moving Average for some time_period (in days)\n", + " EMA = data['Close'].ewm(span=time_period, adjust=False).mean()\n", + " #Calculate the DEMA\n", + " DEMA = 2*EMA - EMA.ewm(span=time_period, adjust=False).mean()\n", + " return DEMA\n", + "\n", + "def upper_shadow(df):\n", + " return df['High'] - np.maximum(df['Close'], df['Open'])\n", + "\n", + "def lower_shadow(df):\n", + " return np.minimum(df['Close'], df['Open']) - df['Low']" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Feature Engineering" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "tags": [ + "block:feature_engineering", + "prev:load_data" + ] + }, + "outputs": [], + "source": [ + "def get_features(df, \n", + " asset_id, \n", + " train=True):\n", + " '''\n", + " This function takes a dataframe with all asset data and return the lagged features for a single asset.\n", + " \n", + " df - Full dataframe with all assets included\n", + " asset_id - integer from 0-13 inclusive to represent a cryptocurrency asset\n", + " train - True - you are training your model\n", + " - False - you are submitting your model via api\n", + " '''\n", + " # filter based on asset id\n", + " df = df[df['Asset_ID']==asset_id]\n", + " \n", + " # sort based on time stamp\n", + " df = df.sort_values('timestamp')\n", + " \n", + " if train == True:\n", + " df_feat = df.copy()\n", + " \n", + " # define a train_flg column to split your data into train and validation\n", + " totimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n", + " valid_window = [totimestamp(\"01/05/2021\")]\n", + " \n", + " df_feat['train_flg'] = np.where(df_feat['timestamp']>=valid_window[0], 0,1)\n", + " df_feat = df_feat[['timestamp','Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','Target','train_flg']].copy()\n", + " else:\n", + " df = df.sort_values('row_id')\n", + " df_feat = df[['Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','row_id']].copy()\n", + " \n", + " for i in tqdm([30, 120, 240]):\n", + " # creating technical indicators\n", + " df_feat[f'RSI_{i}'] = RSI(df_feat, i)\n", + " df_feat[f'ATR_{i}'] = ATR(df_feat, i)\n", + " df_feat[f'DEMA_{i}'] = DEMA(df_feat, i)\n", + "\n", + " for i in tqdm([30, 120, 240]):\n", + " # creating lag features\n", + " df_feat[f'sma_{i}'] = df_feat['Close'].rolling(i).mean()/df_feat['Close'] -1\n", + " df_feat[f'return_{i}'] = df_feat['Close']/df_feat['Close'].shift(i) -1\n", + " \n", + " # new featu# creating technical indicators featureses\n", + " df_feat['HL'] = np.log(df_feat['High'] - df_feat['Low'])\n", + " df_feat['OC'] = np.log(df_feat['Close'] - df_feat['Open'])\n", + " \n", + " df_feat['lower_shadow'] = np.log(lower_shadow(df)) \n", + " df_feat['upper_shadow'] = np.log(upper_shadow(df))\n", + " \n", + " # replace inf with nan\n", + " df_feat.replace([np.inf, -np.inf], np.nan, inplace=True)\n", + " \n", + " # datetime features\n", + " df_feat['Date'] = pd.to_datetime(df_feat['timestamp'], unit='s')\n", + " df_feat['Day'] = df_feat['Date'].dt.weekday.astype(np.int32)\n", + " df_feat[\"dayofyear\"] = df_feat['Date'].dt.dayofyear\n", + " df_feat[\"weekofyear\"] = df_feat['Date'].dt.weekofyear\n", + " df_feat[\"season\"] = ((df_feat['Date'].dt.month)%12 + 3)//3\n", + " \n", + "\n", + " df_feat = df_feat.drop(['Open','Close','High','Low', 'Volume', 'Date'], axis=1)\n", + " \n", + " # fill nan values with 0\n", + " df_feat = df_feat.fillna(0)\n", + " \n", + " return df_feat" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 7.64it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 12.66it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 7.61it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 16.32it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 9.89it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 22.05it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 8.52it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 17.92it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 10.35it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 22.56it/s]\n" + ] + } + ], + "source": [ + "# create your feature dataframe for each asset and concatenate\n", + "feature_df = pd.DataFrame()\n", + "for i in range(14):\n", + " print(i)\n", + " feature_df = pd.concat([feature_df,get_features(df_train,i,train=True)])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Merge Assets Features" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:merge_assets_features", + "prev:load_data", + "prev:feature_engineering" + ] + }, + "outputs": [], + "source": [ + "# assign weight column feature dataframe\n", + "feature_df = pd.merge(feature_df, df_asset_details[['Asset_ID','Weight']], how='left', on=['Asset_ID'])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "feature_df.columns" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Modelling" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:modelling", + "prev:merge_assets_features" + ] + }, + "outputs": [], + "source": [ + "# define features for LGBM\n", + "features = ['Asset_ID', 'RSI_30', 'ATR_30',\n", + " 'DEMA_30', 'RSI_120', 'ATR_120', 'DEMA_120', 'RSI_240', 'ATR_240',\n", + " 'DEMA_240', 'sma_30', 'return_30', 'sma_120', 'return_120', 'sma_240',\n", + " 'return_240', 'HL', 'OC', 'lower_shadow', 'upper_shadow', 'Day',\n", + " 'dayofyear', 'weekofyear', 'season']\n", + "categoricals = ['Asset_ID']" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# define train and validation weights and datasets\n", + "weights_train = feature_df.query('train_flg == 1')[['Weight']]\n", + "weights_test = feature_df.query('train_flg == 0')[['Weight']]\n", + "\n", + "train_dataset = lgb.Dataset(feature_df.query('train_flg == 1')[features], \n", + " feature_df.query('train_flg == 1')['Target'].values, \n", + " feature_name = features,\n", + " categorical_feature= categoricals)\n", + "val_dataset = lgb.Dataset(feature_df.query('train_flg == 0')[features], \n", + " feature_df.query('train_flg == 0')['Target'].values, \n", + " feature_name = features,\n", + " categorical_feature= categoricals)\n", + "\n", + "train_dataset.add_w = weights_train\n", + "val_dataset.add_w = weights_test\n", + "\n", + "evals_result = {}\n", + "params = {'n_estimators': int(N_EST),\n", + " 'objective': 'regression',\n", + " 'metric': 'rmse',\n", + " 'boosting_type': 'gbdt',\n", + " 'max_depth': -1, \n", + " 'learning_rate': float(LR),\n", + " 'seed': 2022,\n", + " 'verbose': -1,\n", + " }\n", + "\n", + "# train LGBM2\n", + "model = lgb.train(params = params,\n", + " train_set = train_dataset, \n", + " valid_sets = [val_dataset],\n", + " early_stopping_rounds=60,\n", + " verbose_eval = 30,\n", + " feval=weighted_correlation,\n", + " evals_result = evals_result \n", + " )\n", + "\n", + "joblib.dump(model, 'lgb.jl')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "fea_imp = pd.DataFrame({'imp':model.feature_importance(), 'col': features})\n", + "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", + "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Evaluation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:evaluation_result", + "prev:modelling" + ] + }, + "outputs": [], + "source": [ + "model = joblib.load('lgb.jl')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "root_mean_squared_error = model.best_score.get('valid_0').get('rmse')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "weighted_correlation = model.best_score.get('valid_0').get('eval_wcorr')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Pipeline Metrics" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [], + "source": [ + "print(root_mean_squared_error)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [], + "source": [ + "print(weighted_correlation)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "kubeflow_notebook": { + "autosnapshot": true, + "experiment": { + "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", + "name": "g-research-crypto-forecasting" + }, + "experiment_name": "g-research-crypto-forecasting", + "katib_metadata": { + "algorithm": { + "algorithmName": "grid" + }, + "maxFailedTrialCount": 3, + "maxTrialCount": 12, + "objective": { + "objectiveMetricName": "", + "type": "minimize" + }, + "parallelTrialCount": 3, + "parameters": [] + }, + "katib_run": false, + "pipeline_description": "forecasting short term returns in 14 popular cryptocurrencies.", + "pipeline_name": "g-research-crypto-forecasting-pipeline", + "snapshot_volumes": true, + "steps_defaults": [ + "label:access-ml-pipeline:true", + "label:kaggle-secret:true", + "label:access-rok:true" + ], + "volume_access_mode": "rwm", + "volumes": [ + { + "annotations": [], + "mount_point": "/home/jovyan", + "name": "demo-workspace-fb99v", + "size": 15, + "size_type": "Gi", + "snapshot": false, + "type": "clone" + } + ] + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kfp.ipynb b/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kfp.ipynb index 393f2d12..ba4ca749 100644 --- a/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kfp.ipynb +++ b/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-kfp.ipynb @@ -1,810 +1,810 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# 🪙 G-Research Crypto Kubeflow Pipeline\n", - "![](./images/vector-blockchain-poster.jpg)\n", - "\n", - "---\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to forecast short term returns in 14 popular cryptocurrencies. The dataset provided contains information on historic trades for several cryptoassets, such as Bitcoin and Ethereum. \n", - "\n", - "> G-Research is a leading quantitative research and technology company. By using the latest scientific techniques, they produce world-beating predictive research and build advanced technology to analyse the world's data." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Install relevant libraries\n", - "\n", - "\n", - ">Update pip `pip install --user --upgrade pip`\n", - "\n", - ">Install and upgrade kubeflow sdk `pip install kfp --upgrade --user --quiet`\n", - "\n", - "You may need to restart your notebook kernel after installing the kfp sdk" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Requirement already satisfied: pip in /usr/local/lib/python3.6/dist-packages (21.3.1)\n" - ] - } - ], - "source": [ - "!pip install --user --upgrade pip" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "!pip install kfp --upgrade --user --quiet" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Name: kfp\n", - "Version: 1.8.11\n", - "Summary: KubeFlow Pipelines SDK\n", - "Home-page: https://github.com/kubeflow/pipelines\n", - "Author: The Kubeflow Authors\n", - "Author-email: \n", - "License: UNKNOWN\n", - "Location: /home/jovyan/.local/lib/python3.6/site-packages\n", - "Requires: absl-py, click, cloudpickle, dataclasses, Deprecated, docstring-parser, fire, google-api-python-client, google-auth, google-cloud-storage, jsonschema, kfp-pipeline-spec, kfp-server-api, kubernetes, protobuf, pydantic, PyYAML, requests-toolbelt, strip-hints, tabulate, typer, typing-extensions, uritemplate\n", - "Required-by: kubeflow-kale\n" - ] - } - ], - "source": [ - "# confirm the kfp sdk\n", - "! pip show kfp" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "tags": [ - "imports" - ] - }, - "outputs": [], - "source": [ - "import kfp\n", - "import kfp.components as comp\n", - "import kfp.dsl as dsl\n", - "from kfp.components import OutputPath\n", - "from typing import NamedTuple" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# Kubeflow pipeline component creation\n", - "\n", - "## Download the dataset" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [], - "source": [ - "# download data step\n", - "def download_data(dataset, \n", - " data_path):\n", - " \n", - " # install the necessary libraries\n", - " import os, sys, subprocess, zipfile, pickle;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','kaggle'])\n", - " \n", - " # import libraries\n", - " import pandas as pd\n", - "\n", - " # setup kaggle environment for data download\n", - " with open('/secret/kaggle-secret/password', 'r') as file:\n", - " kaggle_key = file.read().rstrip()\n", - " with open('/secret/kaggle-secret/username', 'r') as file:\n", - " kaggle_user = file.read().rstrip()\n", - " \n", - " os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", - " \n", - " # create data_path directory\n", - " if not os.path.exists(data_path):\n", - " os.makedirs(data_path)\n", - " \n", - " # download kaggle's g-research-crypto-forecasting data\n", - " subprocess.run([\"kaggle\",\"competitions\", \"download\", \"-c\", dataset])\n", - " \n", - " # extract 'train.csv' and 'asset_details.csv' in g-research-crypto-forecasting.zip to data_path\n", - " with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", - " zip_ref.extractall(data_path, members=['train.csv', 'asset_details.csv'])\n", - " \n", - " return(print('Done!'))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Load Data" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [], - "source": [ - "# load data step\n", - "def load_data(data_path):\n", - " \n", - " # install the necessary libraries\n", - " import os, sys, subprocess, pickle;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " \n", - " # import libraries\n", - " import pandas as pd\n", - "\n", - " TRAIN_CSV = f'{data_path}/train.csv'\n", - " ASSET_DETAILS_CSV = f'{data_path}/asset_details.csv'\n", - " \n", - " # read TRAIN_CSV and ASSET_DETAILS_CSV\n", - " df_train = pd.read_csv(TRAIN_CSV)\n", - " df_asset_details = pd.read_csv(ASSET_DETAILS_CSV).sort_values(\"Asset_ID\")\n", - " \n", - " df_train['datetime'] = pd.to_datetime(df_train['timestamp'], unit='s')\n", - " df_train = df_train[df_train['datetime'] >= '2020-01-01 00:00:00'].copy()\n", - " \n", - " # Save the df_train data as a pickle file to be used by the feature_engineering component.\n", - " with open(f'{data_path}/df_train', 'wb') as f:\n", - " pickle.dump(df_train, f)\n", - " \n", - " # Save the df_train data as a pickle file to be used by the merge_data component.\n", - " with open(f'{data_path}/df_asset_details', 'wb') as g:\n", - " pickle.dump(df_asset_details, g)\n", - "\n", - " \n", - " return(print('Done!'))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Feature Engineering" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "# feature engineering step\n", - "\n", - "def feature_engineering(data_path):\n", - " \n", - " # install the necessary libraries\n", - " import sys, subprocess;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','tqdm'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','talib-binary'])\n", - " \n", - " # import Library\n", - " import os, pickle, time, talib, datetime;\n", - " import numpy as np\n", - " import pandas as pd\n", - " from tqdm import tqdm\n", - "\n", - " # loading the df_train data\n", - " with open(f'{data_path}/df_train', 'rb') as f:\n", - " df_train = pickle.load(f)\n", - " \n", - " # creating technical indicators\n", - " \n", - " # Create a function to calculate the Relative Strength Index\n", - " def RSI(df, n):\n", - " return talib.RSI(df['Close'], n)\n", - " \n", - " # Create a function to calculate the Average True Range\n", - " def ATR(df, n):\n", - " return talib.ATR(df[\"High\"], df.Low, df.Close, n)\n", - "\n", - " # Create a function to calculate the Double Exponential Moving Average (DEMA)\n", - " def DEMA(data, time_period):\n", - " #Calculate the Exponential Moving Average for some time_period (in days)\n", - " EMA = data['Close'].ewm(span=time_period, adjust=False).mean()\n", - " #Calculate the DEMA\n", - " DEMA = 2*EMA - EMA.ewm(span=time_period, adjust=False).mean()\n", - " return DEMA\n", - " \n", - " # Create a function to calculate the upper_shadow\n", - " def upper_shadow(df):\n", - " return df['High'] - np.maximum(df['Close'], df['Open'])\n", - " \n", - " # Create a function to calculate the lower_shadow\n", - " def lower_shadow(df):\n", - " return np.minimum(df['Close'], df['Open']) - df['Low']\n", - " \n", - " \n", - " def get_features(df, asset_id, train=True):\n", - " '''\n", - " This function takes a dataframe with all asset data and return the lagged features for a single asset.\n", - "\n", - " df - Full dataframe with all assets included\n", - " asset_id - integer from 0-13 inclusive to represent a cryptocurrency asset\n", - " train - True - you are training your model\n", - " - False - you are submitting your model via api\n", - " '''\n", - " # filter based on asset id\n", - " df = df[df['Asset_ID']==asset_id]\n", - "\n", - " # sort based on time stamp\n", - " df = df.sort_values('timestamp')\n", - "\n", - " if train == True:\n", - " df_feat = df.copy()\n", - "\n", - " # define a train_flg column to split your data into train and validation\n", - " totimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n", - " valid_window = [totimestamp(\"01/05/2021\")]\n", - "\n", - " df_feat['train_flg'] = np.where(df_feat['timestamp']>=valid_window[0], 0,1)\n", - " df_feat = df_feat[['timestamp','Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','Target','train_flg']].copy()\n", - " else:\n", - " df = df.sort_values('row_id')\n", - " df_feat = df[['Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','row_id']].copy()\n", - "\n", - " for i in tqdm([30, 120, 240]):\n", - " # Applyin technical indicators\n", - " df_feat[f'RSI_{i}'] = RSI(df_feat, i)\n", - " df_feat[f'ATR_{i}'] = ATR(df_feat, i)\n", - " df_feat[f'DEMA_{i}'] = DEMA(df_feat, i)\n", - "\n", - " for i in tqdm([30, 120, 240]):\n", - " # creating lag features\n", - " df_feat[f'sma_{i}'] = df_feat['Close'].rolling(i).mean()/df_feat['Close'] -1\n", - " df_feat[f'return_{i}'] = df_feat['Close']/df_feat['Close'].shift(i) -1\n", - "\n", - " # new features\n", - " df_feat['HL'] = np.log(df_feat['High'] - df_feat['Low'])\n", - " df_feat['OC'] = np.log(df_feat['Close'] - df_feat['Open'])\n", - " \n", - " # Applyin lower_shadow and upper_shadow indicators\n", - " df_feat['lower_shadow'] = np.log(lower_shadow(df)) \n", - " df_feat['upper_shadow'] = np.log(upper_shadow(df))\n", - "\n", - " # replace inf with nan\n", - " df_feat.replace([np.inf, -np.inf], np.nan, inplace=True)\n", - "\n", - " # datetime features\n", - " df_feat['Date'] = pd.to_datetime(df_feat['timestamp'], unit='s')\n", - " df_feat['Day'] = df_feat['Date'].dt.weekday.astype(np.int32)\n", - " df_feat[\"dayofyear\"] = df_feat['Date'].dt.dayofyear\n", - " df_feat[\"weekofyear\"] = df_feat['Date'].dt.weekofyear\n", - " df_feat[\"season\"] = ((df_feat['Date'].dt.month)%12 + 3)//3\n", - " \n", - " # drop features\n", - " df_feat = df_feat.drop(['Open','Close','High','Low', 'Volume', 'Date'], axis=1)\n", - "\n", - " # fill nan values with 0\n", - " df_feat = df_feat.fillna(0)\n", - "\n", - " return df_feat\n", - " \n", - " # create your features dataframe for each asset and concatenate\n", - " feature_df = pd.DataFrame()\n", - " for i in range(14):\n", - " print(i)\n", - " feature_df = pd.concat([feature_df,get_features(df_train,i,train=True)])\n", - " \n", - " # save the feature engineered data as a pickle file to be used by the modeling component.\n", - " with open(f'{data_path}/feature_df', 'wb') as f:\n", - " pickle.dump(feature_df, f)\n", - " \n", - " return(print('Done!')) " - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Merge Assets Data and Features" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [], - "source": [ - "# merge_assets_features step\n", - "\n", - "def merge_assets_features(data_path):\n", - " \n", - " # install the necessary libraries\n", - " import sys, subprocess;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " \n", - " # import Library\n", - " import os, pickle;\n", - " import pandas as pd\n", - "\n", - " #loading the feature_df data\n", - " with open(f'{data_path}/feature_df', 'rb') as f:\n", - " feature_df = pickle.load(f)\n", - " \n", - " #loading the df_asset_details data\n", - " with open(f'{data_path}/df_asset_details', 'rb') as g:\n", - " df_asset_details = pickle.load(g)\n", - " \n", - " # assign weight column feature dataframe\n", - " feature_df = pd.merge(feature_df, df_asset_details[['Asset_ID','Weight']], how='left', on=['Asset_ID'])\n", - "\n", - " #Save the feature_df as a pickle file to be used by the modelling component.\n", - " with open(f'{data_path}/merge_feature_df', 'wb') as h:\n", - " pickle.dump(feature_df, h)\n", - " \n", - " return(print('Done!')) " - ] - }, - { - "cell_type": "markdown", - "metadata": { - "papermill": { - "duration": 0.01421, - "end_time": "2022-04-17T07:17:13.396620", - "exception": false, - "start_time": "2022-04-17T07:17:13.382410", - "status": "completed" - }, - "tags": [] - }, - "source": [ - "## Modelling\n", - " " - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "# modeling step\n", - "\n", - "def modeling(data_path):\n", - " \n", - " # install the necessary libraries\n", - " import sys, subprocess;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", - " \n", - " # import Library\n", - " import os, pickle, joblib;\n", - " import pandas as pd\n", - " import numpy as np\n", - " import lightgbm as lgb\n", - " from lightgbm import LGBMRegressor\n", - "\n", - " #loading the new_feats data\n", - " with open(f'{data_path}/merge_feature_df', 'rb') as f:\n", - " feature_df = pickle.load(f)\n", - " \n", - " # define features for LGBM\n", - " features = ['Asset_ID', 'RSI_30', 'ATR_30',\n", - " 'DEMA_30', 'RSI_120', 'ATR_120', 'DEMA_120', 'RSI_240', 'ATR_240',\n", - " 'DEMA_240', 'sma_30', 'return_30', 'sma_120', 'return_120', 'sma_240',\n", - " 'return_240', 'HL', 'OC', 'lower_shadow', 'upper_shadow', 'Day',\n", - " 'dayofyear', 'weekofyear', 'season']\n", - " categoricals = ['Asset_ID']\n", - " \n", - " # define the evaluation metric\n", - " def weighted_correlation(a, train_data):\n", - "\n", - " weights = train_data.add_w.values.flatten()\n", - " b = train_data.get_label()\n", - "\n", - "\n", - " w = np.ravel(weights)\n", - " a = np.ravel(a)\n", - " b = np.ravel(b)\n", - "\n", - " sum_w = np.sum(w)\n", - " mean_a = np.sum(a * w) / sum_w\n", - " mean_b = np.sum(b * w) / sum_w\n", - " var_a = np.sum(w * np.square(a - mean_a)) / sum_w\n", - " var_b = np.sum(w * np.square(b - mean_b)) / sum_w\n", - "\n", - " cov = np.sum((a * b * w)) / np.sum(w) - mean_a * mean_b\n", - " corr = cov / np.sqrt(var_a * var_b)\n", - "\n", - " return 'eval_wcorr', corr, True\n", - " \n", - " # define train and validation weights and datasets\n", - " weights_train = feature_df.query('train_flg == 1')[['Weight']]\n", - " weights_test = feature_df.query('train_flg == 0')[['Weight']]\n", - "\n", - " train_dataset = lgb.Dataset(feature_df.query('train_flg == 1')[features], \n", - " feature_df.query('train_flg == 1')['Target'].values, \n", - " feature_name = features,\n", - " categorical_feature= categoricals)\n", - " val_dataset = lgb.Dataset(feature_df.query('train_flg == 0')[features], \n", - " feature_df.query('train_flg == 0')['Target'].values, \n", - " feature_name = features,\n", - " categorical_feature= categoricals)\n", - " # add weights\n", - " train_dataset.add_w = weights_train\n", - " val_dataset.add_w = weights_test\n", - " \n", - " # LGBM params\n", - " evals_result = {}\n", - " params = {'n_estimators': 1200,\n", - " 'objective': 'regression',\n", - " 'metric': 'rmse',\n", - " 'boosting_type': 'gbdt',\n", - " 'max_depth': -1, \n", - " 'learning_rate': 0.01,\n", - " 'seed': 2022,\n", - " 'verbose': -1,\n", - " }\n", - "\n", - " # train LGBM\n", - " model = lgb.train(params = params,\n", - " train_set = train_dataset, \n", - " valid_sets = [val_dataset],\n", - " early_stopping_rounds=60,\n", - " verbose_eval = 30,\n", - " feval=weighted_correlation,\n", - " evals_result = evals_result \n", - " )\n", - " \n", - " # saving model\n", - " joblib.dump(model, f'{data_path}/lgb.jl')\n", - " \n", - " return(print('Done!')) " - ] - }, - { - "cell_type": "markdown", - "metadata": { - "papermill": { - "duration": 0.01428, - "end_time": "2022-04-17T07:17:23.959655", - "exception": false, - "start_time": "2022-04-17T07:17:23.945375", - "status": "completed" - }, - "tags": [] - }, - "source": [ - "## Evaluation" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - "# evaluation step\n", - "\n", - "def evaluation_result(data_path, \n", - " metrics_path: OutputPath(str)) -> NamedTuple(\"EvaluationOutput\", [(\"mlpipeline_metrics\", \"Metrics\")]):\n", - " \n", - " # import Library\n", - " import sys, subprocess;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", - " import json;\n", - " from collections import namedtuple\n", - " import joblib\n", - " import lightgbm as lgb\n", - " from lightgbm import LGBMRegressor\n", - " \n", - " # load model\n", - " model = joblib.load(f'{data_path}/lgb.jl')\n", - "\n", - " # model evaluation\n", - " root_mean_squared_error = model.best_score.get('valid_0').get('rmse')\n", - " weighted_correlation = model.best_score.get('valid_0').get('eval_wcorr')\n", - " \n", - " # create kubeflow metric metadata for UI \n", - " metrics = {\n", - " 'metrics': [\n", - " {'name': 'root-mean-squared-error',\n", - " 'numberValue': root_mean_squared_error,\n", - " 'format': 'RAW'},\n", - " {'name': 'weighted-correlation',\n", - " 'numberValue': weighted_correlation,\n", - " 'format': 'RAW'}\n", - " ]\n", - " }\n", - " \n", - "\n", - " with open(metrics_path, \"w\") as f:\n", - " json.dump(metrics, f)\n", - "\n", - " output_tuple = namedtuple(\"EvaluationOutput\", [\"mlpipeline_metrics\"])\n", - "\n", - " return output_tuple(json.dumps(metrics))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Create pipeline components \n", - "\n", - "using `create_component_from_func`" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [], - "source": [ - "# create light weight components\n", - "download_op = comp.create_component_from_func(download_data,base_image=\"python:3.7.1\")\n", - "load_op = comp.create_component_from_func(load_data,base_image=\"python:3.7.1\")\n", - "merge_assets_features_op = comp.create_component_from_func(merge_assets_features,base_image=\"python:3.7.1\")\n", - "feature_eng_op = comp.create_component_from_func(feature_engineering,base_image=\"python:3.7.1\")\n", - "modeling_op = comp.create_component_from_func(modeling, base_image=\"python:3.7.1\")\n", - "evaluation_op = comp.create_component_from_func(evaluation_result, base_image=\"python:3.7.1\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Kubeflow pipeline creation" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [], - "source": [ - "# define pipeline\n", - "@dsl.pipeline(name=\"g-research-crypto-forecasting-pipeline\", \n", - " description=\"Forecasting short term returns in 14 popular cryptocurrencies.\")\n", - "\n", - "# Define parameters to be fed into pipeline\n", - "def g_research_crypto_forecast_pipeline(\n", - " dataset: str,\n", - " data_path: str\n", - " ):\n", - " # Define volume to share data between components.\n", - " vop = dsl.VolumeOp(\n", - " name=\"create_data_volume\",\n", - " resource_name=\"data-volume\", \n", - " size=\"16Gi\", \n", - " modes=dsl.VOLUME_MODE_RWO)\n", - " \n", - " \n", - " # Create download container.\n", - " download_container = download_op(dataset, data_path)\\\n", - " .add_pvolumes({data_path: vop.volume}).add_pod_label(\"kaggle-secret\", \"true\")\n", - " # Create load container.\n", - " load_container = load_op(data_path)\\\n", - " .add_pvolumes({data_path: download_container.pvolume})\n", - " # Create feature engineering container.\n", - " feat_eng_container = feature_eng_op(data_path)\\\n", - " .add_pvolumes({data_path: load_container.pvolume})\n", - " # Create merge_assets_feat container.\n", - " merge_assets_feat_container = merge_assets_features_op(data_path)\\\n", - " .add_pvolumes({data_path: feat_eng_container.pvolume})\n", - " # Create modeling container.\n", - " modeling_container = modeling_op(data_path)\\\n", - " .add_pvolumes({data_path: merge_assets_feat_container.pvolume})\n", - " # Create prediction container.\n", - " evaluation_container = evaluation_op(data_path).add_pvolumes({data_path: modeling_container.pvolume})" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [], - "source": [ - "# create client that would enable communication with the Pipelines API server \n", - "client = kfp.Client()" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [], - "source": [ - "# arguments\n", - "dataset = \"g-research-crypto-forecasting\"\n", - "data_path = \"/mnt\"" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "Experiment details." - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "Run details." - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "pipeline_func = g_research_crypto_forecast_pipeline\n", - "\n", - "experiment_name = 'g_research_crypto_forecast_pipeline_lightweight'\n", - "run_name = pipeline_func.__name__ + ' run'\n", - "\n", - "arguments = {\n", - " \"dataset\": dataset,\n", - " \"data_path\": data_path\n", - " }\n", - "\n", - "# Compile pipeline to generate compressed YAML definition of the pipeline.\n", - "kfp.compiler.Compiler().compile(pipeline_func, \n", - " '{}.zip'.format(experiment_name))\n", - "\n", - "# Submit pipeline directly from pipeline function\n", - "run_result = client.create_run_from_pipeline_func(pipeline_func, \n", - " experiment_name=experiment_name, \n", - " run_name=run_name, \n", - " arguments=arguments\n", - " )\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "kubeflow_notebook": { - "autosnapshot": true, - "experiment": { - "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", - "name": "g-research-crypto-forecasting" - }, - "experiment_name": "g-research-crypto-forecasting", - "katib_metadata": { - "algorithm": { - "algorithmName": "grid" - }, - "maxFailedTrialCount": 3, - "maxTrialCount": 12, - "objective": { - "objectiveMetricName": "", - "type": "minimize" - }, - "parallelTrialCount": 3, - "parameters": [] - }, - "katib_run": false, - "pipeline_description": "Forecasting short term returns in 14 popular cryptocurrencies.", - "pipeline_name": "g-research-crypto-forecasting-pipeline", - "snapshot_volumes": true, - "steps_defaults": [ - "label:access-ml-pipeline:true", - "label:kaggle-secret:true", - "label:access-rok:true" - ], - "volume_access_mode": "rwm", - "volumes": [ - { - "annotations": [], - "mount_point": "/home/jovyan", - "name": "demo-workspace-fb99v", - "size": 15, - "size_type": "Gi", - "snapshot": false, - "type": "clone" - } - ] - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - }, - "papermill": { - "default_parameters": {}, - "duration": 32.012084, - "end_time": "2022-04-17T07:17:25.053666", - "environment_variables": {}, - "exception": null, - "input_path": "__notebook__.ipynb", - "output_path": "__notebook__.ipynb", - "parameters": {}, - "start_time": "2022-04-17T07:16:53.041582", - "version": "2.3.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# 🪙 G-Research Crypto Kubeflow Pipeline\n", + "![](./images/vector-blockchain-poster.jpg)\n", + "\n", + "---\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to forecast short term returns in 14 popular cryptocurrencies. The dataset provided contains information on historic trades for several cryptoassets, such as Bitcoin and Ethereum. \n", + "\n", + "> G-Research is a leading quantitative research and technology company. By using the latest scientific techniques, they produce world-beating predictive research and build advanced technology to analyse the world's data." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Install relevant libraries\n", + "\n", + "\n", + ">Update pip `pip install --user --upgrade pip`\n", + "\n", + ">Install and upgrade kubeflow sdk `pip install kfp --upgrade --user --quiet`\n", + "\n", + "You may need to restart your notebook kernel after installing the kfp sdk" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Requirement already satisfied: pip in /usr/local/lib/python3.6/dist-packages (21.3.1)\n" + ] + } + ], + "source": [ + "!pip install --user --upgrade pip" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install kfp --upgrade --user --quiet" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Name: kfp\n", + "Version: 1.8.11\n", + "Summary: KubeFlow Pipelines SDK\n", + "Home-page: https://github.com/kubeflow/pipelines\n", + "Author: The Kubeflow Authors\n", + "Author-email: \n", + "License: UNKNOWN\n", + "Location: /home/jovyan/.local/lib/python3.6/site-packages\n", + "Requires: absl-py, click, cloudpickle, dataclasses, Deprecated, docstring-parser, fire, google-api-python-client, google-auth, google-cloud-storage, jsonschema, kfp-pipeline-spec, kfp-server-api, kubernetes, protobuf, pydantic, PyYAML, requests-toolbelt, strip-hints, tabulate, typer, typing-extensions, uritemplate\n", + "Required-by: kubeflow-kale\n" + ] + } + ], + "source": [ + "# confirm the kfp sdk\n", + "! pip show kfp" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "imports" + ] + }, + "outputs": [], + "source": [ + "import kfp\n", + "import kfp.components as comp\n", + "import kfp.dsl as dsl\n", + "from kfp.components import OutputPath\n", + "from typing import NamedTuple" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# Kubeflow pipeline component creation\n", + "\n", + "## Download the dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "# download data step\n", + "def download_data(dataset, \n", + " data_path):\n", + " \n", + " # install the necessary libraries\n", + " import os, sys, subprocess, zipfile, pickle;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','kaggle'])\n", + " \n", + " # import libraries\n", + " import pandas as pd\n", + "\n", + " # setup kaggle environment for data download\n", + " with open('/secret/kaggle-secret/password', 'r') as file:\n", + " kaggle_key = file.read().rstrip()\n", + " with open('/secret/kaggle-secret/username', 'r') as file:\n", + " kaggle_user = file.read().rstrip()\n", + " \n", + " os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", + " \n", + " # create data_path directory\n", + " if not os.path.exists(data_path):\n", + " os.makedirs(data_path)\n", + " \n", + " # download kaggle's g-research-crypto-forecasting data\n", + " subprocess.run([\"kaggle\",\"competitions\", \"download\", \"-c\", dataset])\n", + " \n", + " # extract 'train.csv' and 'asset_details.csv' in g-research-crypto-forecasting.zip to data_path\n", + " with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", + " zip_ref.extractall(data_path, members=['train.csv', 'asset_details.csv'])\n", + " \n", + " return(print('Done!'))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Load Data" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "# load data step\n", + "def load_data(data_path):\n", + " \n", + " # install the necessary libraries\n", + " import os, sys, subprocess, pickle;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " \n", + " # import libraries\n", + " import pandas as pd\n", + "\n", + " TRAIN_CSV = f'{data_path}/train.csv'\n", + " ASSET_DETAILS_CSV = f'{data_path}/asset_details.csv'\n", + " \n", + " # read TRAIN_CSV and ASSET_DETAILS_CSV\n", + " df_train = pd.read_csv(TRAIN_CSV)\n", + " df_asset_details = pd.read_csv(ASSET_DETAILS_CSV).sort_values(\"Asset_ID\")\n", + " \n", + " df_train['datetime'] = pd.to_datetime(df_train['timestamp'], unit='s')\n", + " df_train = df_train[df_train['datetime'] >= '2020-01-01 00:00:00'].copy()\n", + " \n", + " # Save the df_train data as a pickle file to be used by the feature_engineering component.\n", + " with open(f'{data_path}/df_train', 'wb') as f:\n", + " pickle.dump(df_train, f)\n", + " \n", + " # Save the df_train data as a pickle file to be used by the merge_data component.\n", + " with open(f'{data_path}/df_asset_details', 'wb') as g:\n", + " pickle.dump(df_asset_details, g)\n", + "\n", + " \n", + " return(print('Done!'))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Feature Engineering" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "# feature engineering step\n", + "\n", + "def feature_engineering(data_path):\n", + " \n", + " # install the necessary libraries\n", + " import sys, subprocess;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','tqdm'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','talib-binary'])\n", + " \n", + " # import Library\n", + " import os, pickle, time, talib, datetime;\n", + " import numpy as np\n", + " import pandas as pd\n", + " from tqdm import tqdm\n", + "\n", + " # loading the df_train data\n", + " with open(f'{data_path}/df_train', 'rb') as f:\n", + " df_train = pickle.load(f)\n", + " \n", + " # creating technical indicators\n", + " \n", + " # Create a function to calculate the Relative Strength Index\n", + " def RSI(df, n):\n", + " return talib.RSI(df['Close'], n)\n", + " \n", + " # Create a function to calculate the Average True Range\n", + " def ATR(df, n):\n", + " return talib.ATR(df[\"High\"], df.Low, df.Close, n)\n", + "\n", + " # Create a function to calculate the Double Exponential Moving Average (DEMA)\n", + " def DEMA(data, time_period):\n", + " #Calculate the Exponential Moving Average for some time_period (in days)\n", + " EMA = data['Close'].ewm(span=time_period, adjust=False).mean()\n", + " #Calculate the DEMA\n", + " DEMA = 2*EMA - EMA.ewm(span=time_period, adjust=False).mean()\n", + " return DEMA\n", + " \n", + " # Create a function to calculate the upper_shadow\n", + " def upper_shadow(df):\n", + " return df['High'] - np.maximum(df['Close'], df['Open'])\n", + " \n", + " # Create a function to calculate the lower_shadow\n", + " def lower_shadow(df):\n", + " return np.minimum(df['Close'], df['Open']) - df['Low']\n", + " \n", + " \n", + " def get_features(df, asset_id, train=True):\n", + " '''\n", + " This function takes a dataframe with all asset data and return the lagged features for a single asset.\n", + "\n", + " df - Full dataframe with all assets included\n", + " asset_id - integer from 0-13 inclusive to represent a cryptocurrency asset\n", + " train - True - you are training your model\n", + " - False - you are submitting your model via api\n", + " '''\n", + " # filter based on asset id\n", + " df = df[df['Asset_ID']==asset_id]\n", + "\n", + " # sort based on time stamp\n", + " df = df.sort_values('timestamp')\n", + "\n", + " if train == True:\n", + " df_feat = df.copy()\n", + "\n", + " # define a train_flg column to split your data into train and validation\n", + " totimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n", + " valid_window = [totimestamp(\"01/05/2021\")]\n", + "\n", + " df_feat['train_flg'] = np.where(df_feat['timestamp']>=valid_window[0], 0,1)\n", + " df_feat = df_feat[['timestamp','Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','Target','train_flg']].copy()\n", + " else:\n", + " df = df.sort_values('row_id')\n", + " df_feat = df[['Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','row_id']].copy()\n", + "\n", + " for i in tqdm([30, 120, 240]):\n", + " # Applyin technical indicators\n", + " df_feat[f'RSI_{i}'] = RSI(df_feat, i)\n", + " df_feat[f'ATR_{i}'] = ATR(df_feat, i)\n", + " df_feat[f'DEMA_{i}'] = DEMA(df_feat, i)\n", + "\n", + " for i in tqdm([30, 120, 240]):\n", + " # creating lag features\n", + " df_feat[f'sma_{i}'] = df_feat['Close'].rolling(i).mean()/df_feat['Close'] -1\n", + " df_feat[f'return_{i}'] = df_feat['Close']/df_feat['Close'].shift(i) -1\n", + "\n", + " # new features\n", + " df_feat['HL'] = np.log(df_feat['High'] - df_feat['Low'])\n", + " df_feat['OC'] = np.log(df_feat['Close'] - df_feat['Open'])\n", + " \n", + " # Applyin lower_shadow and upper_shadow indicators\n", + " df_feat['lower_shadow'] = np.log(lower_shadow(df)) \n", + " df_feat['upper_shadow'] = np.log(upper_shadow(df))\n", + "\n", + " # replace inf with nan\n", + " df_feat.replace([np.inf, -np.inf], np.nan, inplace=True)\n", + "\n", + " # datetime features\n", + " df_feat['Date'] = pd.to_datetime(df_feat['timestamp'], unit='s')\n", + " df_feat['Day'] = df_feat['Date'].dt.weekday.astype(np.int32)\n", + " df_feat[\"dayofyear\"] = df_feat['Date'].dt.dayofyear\n", + " df_feat[\"weekofyear\"] = df_feat['Date'].dt.weekofyear\n", + " df_feat[\"season\"] = ((df_feat['Date'].dt.month)%12 + 3)//3\n", + " \n", + " # drop features\n", + " df_feat = df_feat.drop(['Open','Close','High','Low', 'Volume', 'Date'], axis=1)\n", + "\n", + " # fill nan values with 0\n", + " df_feat = df_feat.fillna(0)\n", + "\n", + " return df_feat\n", + " \n", + " # create your features dataframe for each asset and concatenate\n", + " feature_df = pd.DataFrame()\n", + " for i in range(14):\n", + " print(i)\n", + " feature_df = pd.concat([feature_df,get_features(df_train,i,train=True)])\n", + " \n", + " # save the feature engineered data as a pickle file to be used by the modeling component.\n", + " with open(f'{data_path}/feature_df', 'wb') as f:\n", + " pickle.dump(feature_df, f)\n", + " \n", + " return(print('Done!')) " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Merge Assets Data and Features" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "# merge_assets_features step\n", + "\n", + "def merge_assets_features(data_path):\n", + " \n", + " # install the necessary libraries\n", + " import sys, subprocess;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " \n", + " # import Library\n", + " import os, pickle;\n", + " import pandas as pd\n", + "\n", + " #loading the feature_df data\n", + " with open(f'{data_path}/feature_df', 'rb') as f:\n", + " feature_df = pickle.load(f)\n", + " \n", + " #loading the df_asset_details data\n", + " with open(f'{data_path}/df_asset_details', 'rb') as g:\n", + " df_asset_details = pickle.load(g)\n", + " \n", + " # assign weight column feature dataframe\n", + " feature_df = pd.merge(feature_df, df_asset_details[['Asset_ID','Weight']], how='left', on=['Asset_ID'])\n", + "\n", + " #Save the feature_df as a pickle file to be used by the modelling component.\n", + " with open(f'{data_path}/merge_feature_df', 'wb') as h:\n", + " pickle.dump(feature_df, h)\n", + " \n", + " return(print('Done!')) " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.01421, + "end_time": "2022-04-17T07:17:13.396620", + "exception": false, + "start_time": "2022-04-17T07:17:13.382410", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## Modelling\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "# modeling step\n", + "\n", + "def modeling(data_path):\n", + " \n", + " # install the necessary libraries\n", + " import sys, subprocess;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", + " \n", + " # import Library\n", + " import os, pickle, joblib;\n", + " import pandas as pd\n", + " import numpy as np\n", + " import lightgbm as lgb\n", + " from lightgbm import LGBMRegressor\n", + "\n", + " #loading the new_feats data\n", + " with open(f'{data_path}/merge_feature_df', 'rb') as f:\n", + " feature_df = pickle.load(f)\n", + " \n", + " # define features for LGBM\n", + " features = ['Asset_ID', 'RSI_30', 'ATR_30',\n", + " 'DEMA_30', 'RSI_120', 'ATR_120', 'DEMA_120', 'RSI_240', 'ATR_240',\n", + " 'DEMA_240', 'sma_30', 'return_30', 'sma_120', 'return_120', 'sma_240',\n", + " 'return_240', 'HL', 'OC', 'lower_shadow', 'upper_shadow', 'Day',\n", + " 'dayofyear', 'weekofyear', 'season']\n", + " categoricals = ['Asset_ID']\n", + " \n", + " # define the evaluation metric\n", + " def weighted_correlation(a, train_data):\n", + "\n", + " weights = train_data.add_w.values.flatten()\n", + " b = train_data.get_label()\n", + "\n", + "\n", + " w = np.ravel(weights)\n", + " a = np.ravel(a)\n", + " b = np.ravel(b)\n", + "\n", + " sum_w = np.sum(w)\n", + " mean_a = np.sum(a * w) / sum_w\n", + " mean_b = np.sum(b * w) / sum_w\n", + " var_a = np.sum(w * np.square(a - mean_a)) / sum_w\n", + " var_b = np.sum(w * np.square(b - mean_b)) / sum_w\n", + "\n", + " cov = np.sum((a * b * w)) / np.sum(w) - mean_a * mean_b\n", + " corr = cov / np.sqrt(var_a * var_b)\n", + "\n", + " return 'eval_wcorr', corr, True\n", + " \n", + " # define train and validation weights and datasets\n", + " weights_train = feature_df.query('train_flg == 1')[['Weight']]\n", + " weights_test = feature_df.query('train_flg == 0')[['Weight']]\n", + "\n", + " train_dataset = lgb.Dataset(feature_df.query('train_flg == 1')[features], \n", + " feature_df.query('train_flg == 1')['Target'].values, \n", + " feature_name = features,\n", + " categorical_feature= categoricals)\n", + " val_dataset = lgb.Dataset(feature_df.query('train_flg == 0')[features], \n", + " feature_df.query('train_flg == 0')['Target'].values, \n", + " feature_name = features,\n", + " categorical_feature= categoricals)\n", + " # add weights\n", + " train_dataset.add_w = weights_train\n", + " val_dataset.add_w = weights_test\n", + " \n", + " # LGBM params\n", + " evals_result = {}\n", + " params = {'n_estimators': 1200,\n", + " 'objective': 'regression',\n", + " 'metric': 'rmse',\n", + " 'boosting_type': 'gbdt',\n", + " 'max_depth': -1, \n", + " 'learning_rate': 0.01,\n", + " 'seed': 2022,\n", + " 'verbose': -1,\n", + " }\n", + "\n", + " # train LGBM\n", + " model = lgb.train(params = params,\n", + " train_set = train_dataset, \n", + " valid_sets = [val_dataset],\n", + " early_stopping_rounds=60,\n", + " verbose_eval = 30,\n", + " feval=weighted_correlation,\n", + " evals_result = evals_result \n", + " )\n", + " \n", + " # saving model\n", + " joblib.dump(model, f'{data_path}/lgb.jl')\n", + " \n", + " return(print('Done!')) " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.01428, + "end_time": "2022-04-17T07:17:23.959655", + "exception": false, + "start_time": "2022-04-17T07:17:23.945375", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## Evaluation" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "# evaluation step\n", + "\n", + "def evaluation_result(data_path, \n", + " metrics_path: OutputPath(str)) -> NamedTuple(\"EvaluationOutput\", [(\"mlpipeline_metrics\", \"Metrics\")]):\n", + " \n", + " # import Library\n", + " import sys, subprocess;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", + " import json;\n", + " from collections import namedtuple\n", + " import joblib\n", + " import lightgbm as lgb\n", + " from lightgbm import LGBMRegressor\n", + " \n", + " # load model\n", + " model = joblib.load(f'{data_path}/lgb.jl')\n", + "\n", + " # model evaluation\n", + " root_mean_squared_error = model.best_score.get('valid_0').get('rmse')\n", + " weighted_correlation = model.best_score.get('valid_0').get('eval_wcorr')\n", + " \n", + " # create kubeflow metric metadata for UI \n", + " metrics = {\n", + " 'metrics': [\n", + " {'name': 'root-mean-squared-error',\n", + " 'numberValue': root_mean_squared_error,\n", + " 'format': 'RAW'},\n", + " {'name': 'weighted-correlation',\n", + " 'numberValue': weighted_correlation,\n", + " 'format': 'RAW'}\n", + " ]\n", + " }\n", + " \n", + "\n", + " with open(metrics_path, \"w\") as f:\n", + " json.dump(metrics, f)\n", + "\n", + " output_tuple = namedtuple(\"EvaluationOutput\", [\"mlpipeline_metrics\"])\n", + "\n", + " return output_tuple(json.dumps(metrics))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create pipeline components \n", + "\n", + "using `create_component_from_func`" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "# create light weight components\n", + "download_op = comp.create_component_from_func(download_data,base_image=\"python:3.7.1\")\n", + "load_op = comp.create_component_from_func(load_data,base_image=\"python:3.7.1\")\n", + "merge_assets_features_op = comp.create_component_from_func(merge_assets_features,base_image=\"python:3.7.1\")\n", + "feature_eng_op = comp.create_component_from_func(feature_engineering,base_image=\"python:3.7.1\")\n", + "modeling_op = comp.create_component_from_func(modeling, base_image=\"python:3.7.1\")\n", + "evaluation_op = comp.create_component_from_func(evaluation_result, base_image=\"python:3.7.1\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Kubeflow pipeline creation" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "# define pipeline\n", + "@dsl.pipeline(name=\"g-research-crypto-forecasting-pipeline\", \n", + " description=\"Forecasting short term returns in 14 popular cryptocurrencies.\")\n", + "\n", + "# Define parameters to be fed into pipeline\n", + "def g_research_crypto_forecast_pipeline(\n", + " dataset: str,\n", + " data_path: str\n", + " ):\n", + " # Define volume to share data between components.\n", + " vop = dsl.VolumeOp(\n", + " name=\"create_data_volume\",\n", + " resource_name=\"data-volume\", \n", + " size=\"16Gi\", \n", + " modes=dsl.VOLUME_MODE_RWO)\n", + " \n", + " \n", + " # Create download container.\n", + " download_container = download_op(dataset, data_path)\\\n", + " .add_pvolumes({data_path: vop.volume}).add_pod_label(\"kaggle-secret\", \"true\")\n", + " # Create load container.\n", + " load_container = load_op(data_path)\\\n", + " .add_pvolumes({data_path: download_container.pvolume})\n", + " # Create feature engineering container.\n", + " feat_eng_container = feature_eng_op(data_path)\\\n", + " .add_pvolumes({data_path: load_container.pvolume})\n", + " # Create merge_assets_feat container.\n", + " merge_assets_feat_container = merge_assets_features_op(data_path)\\\n", + " .add_pvolumes({data_path: feat_eng_container.pvolume})\n", + " # Create modeling container.\n", + " modeling_container = modeling_op(data_path)\\\n", + " .add_pvolumes({data_path: merge_assets_feat_container.pvolume})\n", + " # Create prediction container.\n", + " evaluation_container = evaluation_op(data_path).add_pvolumes({data_path: modeling_container.pvolume})" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "# create client that would enable communication with the Pipelines API server \n", + "client = kfp.Client()" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "# arguments\n", + "dataset = \"g-research-crypto-forecasting\"\n", + "data_path = \"/mnt\"" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "Experiment details." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "Run details." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "pipeline_func = g_research_crypto_forecast_pipeline\n", + "\n", + "experiment_name = 'g_research_crypto_forecast_pipeline_lightweight'\n", + "run_name = pipeline_func.__name__ + ' run'\n", + "\n", + "arguments = {\n", + " \"dataset\": dataset,\n", + " \"data_path\": data_path\n", + " }\n", + "\n", + "# Compile pipeline to generate compressed YAML definition of the pipeline.\n", + "kfp.compiler.Compiler().compile(pipeline_func, \n", + " '{}.zip'.format(experiment_name))\n", + "\n", + "# Submit pipeline directly from pipeline function\n", + "run_result = client.create_run_from_pipeline_func(pipeline_func, \n", + " experiment_name=experiment_name, \n", + " run_name=run_name, \n", + " arguments=arguments\n", + " )\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "kubeflow_notebook": { + "autosnapshot": true, + "experiment": { + "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", + "name": "g-research-crypto-forecasting" + }, + "experiment_name": "g-research-crypto-forecasting", + "katib_metadata": { + "algorithm": { + "algorithmName": "grid" + }, + "maxFailedTrialCount": 3, + "maxTrialCount": 12, + "objective": { + "objectiveMetricName": "", + "type": "minimize" + }, + "parallelTrialCount": 3, + "parameters": [] + }, + "katib_run": false, + "pipeline_description": "Forecasting short term returns in 14 popular cryptocurrencies.", + "pipeline_name": "g-research-crypto-forecasting-pipeline", + "snapshot_volumes": true, + "steps_defaults": [ + "label:access-ml-pipeline:true", + "label:kaggle-secret:true", + "label:access-rok:true" + ], + "volume_access_mode": "rwm", + "volumes": [ + { + "annotations": [], + "mount_point": "/home/jovyan", + "name": "demo-workspace-fb99v", + "size": 15, + "size_type": "Gi", + "snapshot": false, + "type": "clone" + } + ] + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + }, + "papermill": { + "default_parameters": {}, + "duration": 32.012084, + "end_time": "2022-04-17T07:17:25.053666", + "environment_variables": {}, + "exception": null, + "input_path": "__notebook__.ipynb", + "output_path": "__notebook__.ipynb", + "parameters": {}, + "start_time": "2022-04-17T07:16:53.041582", + "version": "2.3.4" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-orig.ipynb b/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-orig.ipynb index f2e89f3e..8245400c 100644 --- a/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-orig.ipynb +++ b/G-research-crypto-forecasting-kaggle-competition/g-research-crypto-forecast-orig.ipynb @@ -1,1013 +1,1013 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# 🪙 G-Research Crypto Original Notebook\n", - "![](./images/vector-blockchain-poster.jpg)\n", - "\n", - "---\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to forecast short term returns in 14 popular cryptocurrencies. The dataset provided contains information on historic trades for several cryptoassets, such as Bitcoin and Ethereum. \n", - "\n", - "> G-Research is a leading quantitative research and technology company. By using the latest scientific techniques, they produce world-beating predictive research and build advanced technology to analyse the world's data." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Install necessary packages\n", - "\n", - "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", - "\n", - "NOTE: After installing python packages, restart notebook kernel before proceeding." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "!pip install -r requirements.txt --user --quiet" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Imports\n", - "\n", - "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "tags": [ - "imports" - ] - }, - "outputs": [], - "source": [ - "import os, random, subprocess\n", - "import pandas as pd\n", - "import numpy as np\n", - "import time, datetime, zipfile\n", - "import joblib, talib\n", - "from tqdm import tqdm\n", - "import lightgbm as lgb\n", - "\n", - "import warnings\n", - "warnings.filterwarnings(\"ignore\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Project hyper-parameters\n", - "\n", - "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "tags": [ - "pipeline-parameters" - ] - }, - "outputs": [], - "source": [ - "# Hyper-parameters\n", - "LR = 0.01\n", - "N_EST = 1200" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "Set random seed for reproducibility" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "def fix_all_seeds(seed):\n", - " np.random.seed(seed)\n", - " random.seed(seed)\n", - " os.environ['PYTHONHASHSEED'] = str(seed)\n", - "\n", - "fix_all_seeds(2022)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Download data\n", - "\n", - "In this section, we download the data from kaggle using the Kaggle API credentials" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "tags": [ - "block:download_data" - ] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "CompletedProcess(args=['kaggle', 'competitions', 'download', '-c', 'g-research-crypto-forecasting'], returncode=0)" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# setup kaggle environment for data download\n", - "dataset = \"g-research-crypto-forecasting\"\n", - "\n", - "# setup kaggle environment for data download\n", - "with open('/secret/kaggle-secret/password', 'r') as file:\n", - " kaggle_key = file.read().rstrip()\n", - "with open('/secret/kaggle-secret/username', 'r') as file:\n", - " kaggle_user = file.read().rstrip()\n", - "\n", - "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", - "\n", - "# download kaggle's g-research-crypto-forecast data\n", - "subprocess.run([\"kaggle\",\"competitions\", \"download\", \"-c\", dataset])" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "tags": [ - "block:" - ] - }, - "outputs": [], - "source": [ - "# path to download to\n", - "data_path = 'data'\n", - "\n", - "# extract g-research-crypto-forecasting.zip to load_data_path\n", - "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", - " zip_ref.extractall(data_path, members=['train.csv', 'asset_details.csv'])" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Load the dataset\n", - "\n", - "First, let us load and analyze the data.\n", - "\n", - "The data is in csv format, thus, we use the handy read_csv pandas method." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "tags": [ - "block:load_data", - "prev:download_data" - ] - }, - "outputs": [], - "source": [ - "TRAIN_CSV = f'{data_path}/train.csv'\n", - "ASSET_DETAILS_CSV = f'{data_path}/asset_details.csv'" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "df_train = pd.read_csv(TRAIN_CSV)" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "(24236806, 10)" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Asset_IDWeightAsset_Name
104.304065Binance Coin
216.779922Bitcoin
022.397895Bitcoin Cash
1034.406719Cardano
1343.555348Dogecoin
351.386294EOS.IO
565.894403Ethereum
472.079442Ethereum Classic
1181.098612IOTA
692.397895Litecoin
12101.098612Maker
7111.609438Monero
9122.079442Stellar
8131.791759TRON
\n", - "
" - ], - "text/plain": [ - " Asset_ID Weight Asset_Name\n", - "1 0 4.304065 Binance Coin\n", - "2 1 6.779922 Bitcoin\n", - "0 2 2.397895 Bitcoin Cash\n", - "10 3 4.406719 Cardano\n", - "13 4 3.555348 Dogecoin\n", - "3 5 1.386294 EOS.IO\n", - "5 6 5.894403 Ethereum\n", - "4 7 2.079442 Ethereum Classic\n", - "11 8 1.098612 IOTA\n", - "6 9 2.397895 Litecoin\n", - "12 10 1.098612 Maker\n", - "7 11 1.609438 Monero\n", - "9 12 2.079442 Stellar\n", - "8 13 1.791759 TRON" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_asset_details = pd.read_csv(ASSET_DETAILS_CSV).sort_values(\"Asset_ID\")\n", - "df_asset_details" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "df_train['datetime'] = pd.to_datetime(df_train['timestamp'], unit='s')\n", - "df_train = df_train[df_train['datetime'] >= '2020-01-01 00:00:00'].copy()" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "(12228898, 11)" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "Timestamp('2021-09-21 00:00:00')" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train['datetime'].max()" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "timestamp 0\n", - "Asset_ID 0\n", - "Count 0\n", - "Open 0\n", - "High 0\n", - "Low 0\n", - "Close 0\n", - "Volume 0\n", - "VWAP 9\n", - "Target 262453\n", - "datetime 0\n", - "dtype: int64" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.isna().sum()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "### Define Helper Functions" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "tags": [ - "functions" - ] - }, - "outputs": [], - "source": [ - "# define the evaluation metric\n", - "def weighted_correlation(a, train_data):\n", - " \n", - " weights = train_data.add_w.values.flatten()\n", - " b = train_data.get_label()\n", - " \n", - " \n", - " w = np.ravel(weights)\n", - " a = np.ravel(a)\n", - " b = np.ravel(b)\n", - "\n", - " sum_w = np.sum(w)\n", - " mean_a = np.sum(a * w) / sum_w\n", - " mean_b = np.sum(b * w) / sum_w\n", - " var_a = np.sum(w * np.square(a - mean_a)) / sum_w\n", - " var_b = np.sum(w * np.square(b - mean_b)) / sum_w\n", - "\n", - " cov = np.sum((a * b * w)) / np.sum(w) - mean_a * mean_b\n", - " corr = cov / np.sqrt(var_a * var_b)\n", - "\n", - " return 'eval_wcorr', corr, True" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "tags": [ - "functions" - ] - }, - "outputs": [], - "source": [ - "def RSI(df, n):\n", - " return talib.RSI(df['Close'], n)\n", - "\n", - "def ATR(df, n):\n", - " return talib.ATR(df[\"High\"], df.Low, df.Close, n)\n", - "\n", - "#Create a function to calculate the Double Exponential Moving Average (DEMA)\n", - "def DEMA(data, time_period):\n", - " #Calculate the Exponential Moving Average for some time_period (in days)\n", - " EMA = data['Close'].ewm(span=time_period, adjust=False).mean()\n", - " #Calculate the DEMA\n", - " DEMA = 2*EMA - EMA.ewm(span=time_period, adjust=False).mean()\n", - " return DEMA\n", - "\n", - "def upper_shadow(df):\n", - " return df['High'] - np.maximum(df['Close'], df['Open'])\n", - "\n", - "def lower_shadow(df):\n", - " return np.minimum(df['Close'], df['Open']) - df['Low']" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Feature Engineering" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "tags": [ - "block:feature_engineering", - "prev:load_data" - ] - }, - "outputs": [], - "source": [ - "def get_features(df, \n", - " asset_id, \n", - " train=True):\n", - " '''\n", - " This function takes a dataframe with all asset data and return the lagged features for a single asset.\n", - " \n", - " df - Full dataframe with all assets included\n", - " asset_id - integer from 0-13 inclusive to represent a cryptocurrency asset\n", - " train - True - you are training your model\n", - " - False - you are submitting your model via api\n", - " '''\n", - " # filter based on asset id\n", - " df = df[df['Asset_ID']==asset_id]\n", - " \n", - " # sort based on time stamp\n", - " df = df.sort_values('timestamp')\n", - " \n", - " if train == True:\n", - " df_feat = df.copy()\n", - " \n", - " # define a train_flg column to split your data into train and validation\n", - " totimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n", - " valid_window = [totimestamp(\"01/05/2021\")]\n", - " \n", - " df_feat['train_flg'] = np.where(df_feat['timestamp']>=valid_window[0], 0,1)\n", - " df_feat = df_feat[['timestamp','Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','Target','train_flg']].copy()\n", - " else:\n", - " df = df.sort_values('row_id')\n", - " df_feat = df[['Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','row_id']].copy()\n", - " \n", - " for i in tqdm([30, 120, 240]):\n", - " # creating technical indicators\n", - " df_feat[f'RSI_{i}'] = RSI(df_feat, i)\n", - " df_feat[f'ATR_{i}'] = ATR(df_feat, i)\n", - " df_feat[f'DEMA_{i}'] = DEMA(df_feat, i)\n", - "\n", - " for i in tqdm([30, 120, 240]):\n", - " # creating lag features\n", - " df_feat[f'sma_{i}'] = df_feat['Close'].rolling(i).mean()/df_feat['Close'] -1\n", - " df_feat[f'return_{i}'] = df_feat['Close']/df_feat['Close'].shift(i) -1\n", - " \n", - " # new featu# creating technical indicators featureses\n", - " df_feat['HL'] = np.log(df_feat['High'] - df_feat['Low'])\n", - " df_feat['OC'] = np.log(df_feat['Close'] - df_feat['Open'])\n", - " \n", - " df_feat['lower_shadow'] = np.log(lower_shadow(df)) \n", - " df_feat['upper_shadow'] = np.log(upper_shadow(df))\n", - " \n", - " # replace inf with nan\n", - " df_feat.replace([np.inf, -np.inf], np.nan, inplace=True)\n", - " \n", - " # datetime features\n", - " df_feat['Date'] = pd.to_datetime(df_feat['timestamp'], unit='s')\n", - " df_feat['Day'] = df_feat['Date'].dt.weekday.astype(np.int32)\n", - " df_feat[\"dayofyear\"] = df_feat['Date'].dt.dayofyear\n", - " df_feat[\"weekofyear\"] = df_feat['Date'].dt.weekofyear\n", - " df_feat[\"season\"] = ((df_feat['Date'].dt.month)%12 + 3)//3\n", - "\n", - " df_feat = df_feat.drop(['Open','Close','High','Low', 'Volume', 'Date'], axis=1)\n", - " \n", - " # fill nan values with 0\n", - " df_feat = df_feat.fillna(0)\n", - " return df_feat" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 7.64it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 12.66it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 7.61it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 16.32it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "2\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 9.89it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 22.05it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "3\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 8.52it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 17.92it/s]\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "4\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:00<00:00, 10.35it/s]\n", - "100%|██████████| 3/3 [00:00<00:00, 22.56it/s]\n" - ] - } - ], - "source": [ - "# create your feature dataframe for each asset and concatenate\n", - "feature_df = pd.DataFrame()\n", - "for i in range(14):\n", - " print(i)\n", - " feature_df = pd.concat([feature_df,get_features(df_train,i,train=True)])" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Merge Assets Features" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:merge_assets_features", - "prev:load_data", - "prev:feature_engineering" - ] - }, - "outputs": [], - "source": [ - "# assign weight column feature dataframe\n", - "feature_df = pd.merge(feature_df, df_asset_details[['Asset_ID','Weight']], how='left', on=['Asset_ID'])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "feature_df.columns" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Modelling" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:modelling", - "prev:merge_assets_features" - ] - }, - "outputs": [], - "source": [ - "# define features for LGBM\n", - "features = ['Asset_ID', 'RSI_30', 'ATR_30',\n", - " 'DEMA_30', 'RSI_120', 'ATR_120', 'DEMA_120', 'RSI_240', 'ATR_240',\n", - " 'DEMA_240', 'sma_30', 'return_30', 'sma_120', 'return_120', 'sma_240',\n", - " 'return_240', 'HL', 'OC', 'lower_shadow', 'upper_shadow', 'Day',\n", - " 'dayofyear', 'weekofyear', 'season']\n", - "categoricals = ['Asset_ID']" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# define train and validation weights and datasets\n", - "weights_train = feature_df.query('train_flg == 1')[['Weight']]\n", - "weights_test = feature_df.query('train_flg == 0')[['Weight']]\n", - "\n", - "train_dataset = lgb.Dataset(feature_df.query('train_flg == 1')[features], \n", - " feature_df.query('train_flg == 1')['Target'].values, \n", - " feature_name = features,\n", - " categorical_feature= categoricals)\n", - "val_dataset = lgb.Dataset(feature_df.query('train_flg == 0')[features], \n", - " feature_df.query('train_flg == 0')['Target'].values, \n", - " feature_name = features,\n", - " categorical_feature= categoricals)\n", - "\n", - "train_dataset.add_w = weights_train\n", - "val_dataset.add_w = weights_test\n", - "\n", - "evals_result = {}\n", - "params = {'n_estimators': int(N_EST),\n", - " 'objective': 'regression',\n", - " 'metric': 'rmse',\n", - " 'boosting_type': 'gbdt',\n", - " 'max_depth': -1, \n", - " 'learning_rate': float(LR),\n", - " 'seed': 2022,\n", - " 'verbose': -1,\n", - " }\n", - "\n", - "# train LGBM2\n", - "model = lgb.train(params = params,\n", - " train_set = train_dataset, \n", - " valid_sets = [val_dataset],\n", - " early_stopping_rounds=60,\n", - " verbose_eval = 30,\n", - " feval=weighted_correlation,\n", - " evals_result = evals_result \n", - " )\n", - "\n", - "joblib.dump(model, 'lgb.jl')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "fea_imp = pd.DataFrame({'imp':model.feature_importance(), 'col': features})\n", - "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", - "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Evaluation" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:evaluation_result", - "prev:modelling" - ] - }, - "outputs": [], - "source": [ - "model = joblib.load('lgb.jl')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "root_mean_squared_error = model.best_score.get('valid_0').get('rmse')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "weighted_correlation = model.best_score.get('valid_0').get('eval_wcorr')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [], - "source": [ - "print(root_mean_squared_error)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [], - "source": [ - "print(weighted_correlation)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "kubeflow_notebook": { - "autosnapshot": true, - "experiment": { - "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", - "name": "g-research-crypto-forecasting" - }, - "experiment_name": "g-research-crypto-forecasting", - "katib_metadata": { - "algorithm": { - "algorithmName": "grid" - }, - "maxFailedTrialCount": 3, - "maxTrialCount": 12, - "objective": { - "objectiveMetricName": "", - "type": "minimize" - }, - "parallelTrialCount": 3, - "parameters": [] - }, - "katib_run": false, - "pipeline_description": "forecasting short term returns in 14 popular cryptocurrencies.", - "pipeline_name": "g-research-crypto-forecasting-pipeline", - "snapshot_volumes": true, - "steps_defaults": [ - "label:access-ml-pipeline:true", - "label:kaggle-secret:true", - "label:access-rok:true" - ], - "volume_access_mode": "rwm", - "volumes": [ - { - "annotations": [], - "mount_point": "/home/jovyan", - "name": "test-workspace-6lhtr", - "size": 15, - "size_type": "Gi", - "snapshot": false, - "type": "clone" - } - ] - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# 🪙 G-Research Crypto Original Notebook\n", + "![](./images/vector-blockchain-poster.jpg)\n", + "\n", + "---\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to forecast short term returns in 14 popular cryptocurrencies. The dataset provided contains information on historic trades for several cryptoassets, such as Bitcoin and Ethereum. \n", + "\n", + "> G-Research is a leading quantitative research and technology company. By using the latest scientific techniques, they produce world-beating predictive research and build advanced technology to analyse the world's data." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Install necessary packages\n", + "\n", + "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", + "\n", + "NOTE: After installing python packages, restart notebook kernel before proceeding." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "!pip install -r requirements.txt --user --quiet" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Imports\n", + "\n", + "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "imports" + ] + }, + "outputs": [], + "source": [ + "import os, random, subprocess\n", + "import pandas as pd\n", + "import numpy as np\n", + "import time, datetime, zipfile\n", + "import joblib, talib\n", + "from tqdm import tqdm\n", + "import lightgbm as lgb\n", + "\n", + "import warnings\n", + "warnings.filterwarnings(\"ignore\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Project hyper-parameters\n", + "\n", + "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "pipeline-parameters" + ] + }, + "outputs": [], + "source": [ + "# Hyper-parameters\n", + "LR = 0.01\n", + "N_EST = 1200" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "Set random seed for reproducibility" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "def fix_all_seeds(seed):\n", + " np.random.seed(seed)\n", + " random.seed(seed)\n", + " os.environ['PYTHONHASHSEED'] = str(seed)\n", + "\n", + "fix_all_seeds(2022)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Download data\n", + "\n", + "In this section, we download the data from kaggle using the Kaggle API credentials" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "tags": [ + "block:download_data" + ] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "CompletedProcess(args=['kaggle', 'competitions', 'download', '-c', 'g-research-crypto-forecasting'], returncode=0)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# setup kaggle environment for data download\n", + "dataset = \"g-research-crypto-forecasting\"\n", + "\n", + "# setup kaggle environment for data download\n", + "with open('/secret/kaggle-secret/password', 'r') as file:\n", + " kaggle_key = file.read().rstrip()\n", + "with open('/secret/kaggle-secret/username', 'r') as file:\n", + " kaggle_user = file.read().rstrip()\n", + "\n", + "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", + "\n", + "# download kaggle's g-research-crypto-forecast data\n", + "subprocess.run([\"kaggle\",\"competitions\", \"download\", \"-c\", dataset])" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "tags": [ + "block:" + ] + }, + "outputs": [], + "source": [ + "# path to download to\n", + "data_path = 'data'\n", + "\n", + "# extract g-research-crypto-forecasting.zip to load_data_path\n", + "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", + " zip_ref.extractall(data_path, members=['train.csv', 'asset_details.csv'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Load the dataset\n", + "\n", + "First, let us load and analyze the data.\n", + "\n", + "The data is in csv format, thus, we use the handy read_csv pandas method." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "tags": [ + "block:load_data", + "prev:download_data" + ] + }, + "outputs": [], + "source": [ + "TRAIN_CSV = f'{data_path}/train.csv'\n", + "ASSET_DETAILS_CSV = f'{data_path}/asset_details.csv'" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "df_train = pd.read_csv(TRAIN_CSV)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(24236806, 10)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Asset_IDWeightAsset_Name
104.304065Binance Coin
216.779922Bitcoin
022.397895Bitcoin Cash
1034.406719Cardano
1343.555348Dogecoin
351.386294EOS.IO
565.894403Ethereum
472.079442Ethereum Classic
1181.098612IOTA
692.397895Litecoin
12101.098612Maker
7111.609438Monero
9122.079442Stellar
8131.791759TRON
\n", + "
" + ], + "text/plain": [ + " Asset_ID Weight Asset_Name\n", + "1 0 4.304065 Binance Coin\n", + "2 1 6.779922 Bitcoin\n", + "0 2 2.397895 Bitcoin Cash\n", + "10 3 4.406719 Cardano\n", + "13 4 3.555348 Dogecoin\n", + "3 5 1.386294 EOS.IO\n", + "5 6 5.894403 Ethereum\n", + "4 7 2.079442 Ethereum Classic\n", + "11 8 1.098612 IOTA\n", + "6 9 2.397895 Litecoin\n", + "12 10 1.098612 Maker\n", + "7 11 1.609438 Monero\n", + "9 12 2.079442 Stellar\n", + "8 13 1.791759 TRON" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_asset_details = pd.read_csv(ASSET_DETAILS_CSV).sort_values(\"Asset_ID\")\n", + "df_asset_details" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "df_train['datetime'] = pd.to_datetime(df_train['timestamp'], unit='s')\n", + "df_train = df_train[df_train['datetime'] >= '2020-01-01 00:00:00'].copy()" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(12228898, 11)" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Timestamp('2021-09-21 00:00:00')" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train['datetime'].max()" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "timestamp 0\n", + "Asset_ID 0\n", + "Count 0\n", + "Open 0\n", + "High 0\n", + "Low 0\n", + "Close 0\n", + "Volume 0\n", + "VWAP 9\n", + "Target 262453\n", + "datetime 0\n", + "dtype: int64" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.isna().sum()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "### Define Helper Functions" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "tags": [ + "functions" + ] + }, + "outputs": [], + "source": [ + "# define the evaluation metric\n", + "def weighted_correlation(a, train_data):\n", + " \n", + " weights = train_data.add_w.values.flatten()\n", + " b = train_data.get_label()\n", + " \n", + " \n", + " w = np.ravel(weights)\n", + " a = np.ravel(a)\n", + " b = np.ravel(b)\n", + "\n", + " sum_w = np.sum(w)\n", + " mean_a = np.sum(a * w) / sum_w\n", + " mean_b = np.sum(b * w) / sum_w\n", + " var_a = np.sum(w * np.square(a - mean_a)) / sum_w\n", + " var_b = np.sum(w * np.square(b - mean_b)) / sum_w\n", + "\n", + " cov = np.sum((a * b * w)) / np.sum(w) - mean_a * mean_b\n", + " corr = cov / np.sqrt(var_a * var_b)\n", + "\n", + " return 'eval_wcorr', corr, True" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "tags": [ + "functions" + ] + }, + "outputs": [], + "source": [ + "def RSI(df, n):\n", + " return talib.RSI(df['Close'], n)\n", + "\n", + "def ATR(df, n):\n", + " return talib.ATR(df[\"High\"], df.Low, df.Close, n)\n", + "\n", + "#Create a function to calculate the Double Exponential Moving Average (DEMA)\n", + "def DEMA(data, time_period):\n", + " #Calculate the Exponential Moving Average for some time_period (in days)\n", + " EMA = data['Close'].ewm(span=time_period, adjust=False).mean()\n", + " #Calculate the DEMA\n", + " DEMA = 2*EMA - EMA.ewm(span=time_period, adjust=False).mean()\n", + " return DEMA\n", + "\n", + "def upper_shadow(df):\n", + " return df['High'] - np.maximum(df['Close'], df['Open'])\n", + "\n", + "def lower_shadow(df):\n", + " return np.minimum(df['Close'], df['Open']) - df['Low']" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Feature Engineering" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "tags": [ + "block:feature_engineering", + "prev:load_data" + ] + }, + "outputs": [], + "source": [ + "def get_features(df, \n", + " asset_id, \n", + " train=True):\n", + " '''\n", + " This function takes a dataframe with all asset data and return the lagged features for a single asset.\n", + " \n", + " df - Full dataframe with all assets included\n", + " asset_id - integer from 0-13 inclusive to represent a cryptocurrency asset\n", + " train - True - you are training your model\n", + " - False - you are submitting your model via api\n", + " '''\n", + " # filter based on asset id\n", + " df = df[df['Asset_ID']==asset_id]\n", + " \n", + " # sort based on time stamp\n", + " df = df.sort_values('timestamp')\n", + " \n", + " if train == True:\n", + " df_feat = df.copy()\n", + " \n", + " # define a train_flg column to split your data into train and validation\n", + " totimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n", + " valid_window = [totimestamp(\"01/05/2021\")]\n", + " \n", + " df_feat['train_flg'] = np.where(df_feat['timestamp']>=valid_window[0], 0,1)\n", + " df_feat = df_feat[['timestamp','Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','Target','train_flg']].copy()\n", + " else:\n", + " df = df.sort_values('row_id')\n", + " df_feat = df[['Asset_ID', 'High', 'Low', 'Open', 'Close', 'Volume','row_id']].copy()\n", + " \n", + " for i in tqdm([30, 120, 240]):\n", + " # creating technical indicators\n", + " df_feat[f'RSI_{i}'] = RSI(df_feat, i)\n", + " df_feat[f'ATR_{i}'] = ATR(df_feat, i)\n", + " df_feat[f'DEMA_{i}'] = DEMA(df_feat, i)\n", + "\n", + " for i in tqdm([30, 120, 240]):\n", + " # creating lag features\n", + " df_feat[f'sma_{i}'] = df_feat['Close'].rolling(i).mean()/df_feat['Close'] -1\n", + " df_feat[f'return_{i}'] = df_feat['Close']/df_feat['Close'].shift(i) -1\n", + " \n", + " # new featu# creating technical indicators featureses\n", + " df_feat['HL'] = np.log(df_feat['High'] - df_feat['Low'])\n", + " df_feat['OC'] = np.log(df_feat['Close'] - df_feat['Open'])\n", + " \n", + " df_feat['lower_shadow'] = np.log(lower_shadow(df)) \n", + " df_feat['upper_shadow'] = np.log(upper_shadow(df))\n", + " \n", + " # replace inf with nan\n", + " df_feat.replace([np.inf, -np.inf], np.nan, inplace=True)\n", + " \n", + " # datetime features\n", + " df_feat['Date'] = pd.to_datetime(df_feat['timestamp'], unit='s')\n", + " df_feat['Day'] = df_feat['Date'].dt.weekday.astype(np.int32)\n", + " df_feat[\"dayofyear\"] = df_feat['Date'].dt.dayofyear\n", + " df_feat[\"weekofyear\"] = df_feat['Date'].dt.weekofyear\n", + " df_feat[\"season\"] = ((df_feat['Date'].dt.month)%12 + 3)//3\n", + "\n", + " df_feat = df_feat.drop(['Open','Close','High','Low', 'Volume', 'Date'], axis=1)\n", + " \n", + " # fill nan values with 0\n", + " df_feat = df_feat.fillna(0)\n", + " return df_feat" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 7.64it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 12.66it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 7.61it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 16.32it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 9.89it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 22.05it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 8.52it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 17.92it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:00<00:00, 10.35it/s]\n", + "100%|██████████| 3/3 [00:00<00:00, 22.56it/s]\n" + ] + } + ], + "source": [ + "# create your feature dataframe for each asset and concatenate\n", + "feature_df = pd.DataFrame()\n", + "for i in range(14):\n", + " print(i)\n", + " feature_df = pd.concat([feature_df,get_features(df_train,i,train=True)])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Merge Assets Features" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:merge_assets_features", + "prev:load_data", + "prev:feature_engineering" + ] + }, + "outputs": [], + "source": [ + "# assign weight column feature dataframe\n", + "feature_df = pd.merge(feature_df, df_asset_details[['Asset_ID','Weight']], how='left', on=['Asset_ID'])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "feature_df.columns" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Modelling" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:modelling", + "prev:merge_assets_features" + ] + }, + "outputs": [], + "source": [ + "# define features for LGBM\n", + "features = ['Asset_ID', 'RSI_30', 'ATR_30',\n", + " 'DEMA_30', 'RSI_120', 'ATR_120', 'DEMA_120', 'RSI_240', 'ATR_240',\n", + " 'DEMA_240', 'sma_30', 'return_30', 'sma_120', 'return_120', 'sma_240',\n", + " 'return_240', 'HL', 'OC', 'lower_shadow', 'upper_shadow', 'Day',\n", + " 'dayofyear', 'weekofyear', 'season']\n", + "categoricals = ['Asset_ID']" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# define train and validation weights and datasets\n", + "weights_train = feature_df.query('train_flg == 1')[['Weight']]\n", + "weights_test = feature_df.query('train_flg == 0')[['Weight']]\n", + "\n", + "train_dataset = lgb.Dataset(feature_df.query('train_flg == 1')[features], \n", + " feature_df.query('train_flg == 1')['Target'].values, \n", + " feature_name = features,\n", + " categorical_feature= categoricals)\n", + "val_dataset = lgb.Dataset(feature_df.query('train_flg == 0')[features], \n", + " feature_df.query('train_flg == 0')['Target'].values, \n", + " feature_name = features,\n", + " categorical_feature= categoricals)\n", + "\n", + "train_dataset.add_w = weights_train\n", + "val_dataset.add_w = weights_test\n", + "\n", + "evals_result = {}\n", + "params = {'n_estimators': int(N_EST),\n", + " 'objective': 'regression',\n", + " 'metric': 'rmse',\n", + " 'boosting_type': 'gbdt',\n", + " 'max_depth': -1, \n", + " 'learning_rate': float(LR),\n", + " 'seed': 2022,\n", + " 'verbose': -1,\n", + " }\n", + "\n", + "# train LGBM2\n", + "model = lgb.train(params = params,\n", + " train_set = train_dataset, \n", + " valid_sets = [val_dataset],\n", + " early_stopping_rounds=60,\n", + " verbose_eval = 30,\n", + " feval=weighted_correlation,\n", + " evals_result = evals_result \n", + " )\n", + "\n", + "joblib.dump(model, 'lgb.jl')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "fea_imp = pd.DataFrame({'imp':model.feature_importance(), 'col': features})\n", + "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", + "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Evaluation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:evaluation_result", + "prev:modelling" + ] + }, + "outputs": [], + "source": [ + "model = joblib.load('lgb.jl')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "root_mean_squared_error = model.best_score.get('valid_0').get('rmse')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "weighted_correlation = model.best_score.get('valid_0').get('eval_wcorr')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [], + "source": [ + "print(root_mean_squared_error)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [], + "source": [ + "print(weighted_correlation)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "kubeflow_notebook": { + "autosnapshot": true, + "experiment": { + "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", + "name": "g-research-crypto-forecasting" + }, + "experiment_name": "g-research-crypto-forecasting", + "katib_metadata": { + "algorithm": { + "algorithmName": "grid" + }, + "maxFailedTrialCount": 3, + "maxTrialCount": 12, + "objective": { + "objectiveMetricName": "", + "type": "minimize" + }, + "parallelTrialCount": 3, + "parameters": [] + }, + "katib_run": false, + "pipeline_description": "forecasting short term returns in 14 popular cryptocurrencies.", + "pipeline_name": "g-research-crypto-forecasting-pipeline", + "snapshot_volumes": true, + "steps_defaults": [ + "label:access-ml-pipeline:true", + "label:kaggle-secret:true", + "label:access-rok:true" + ], + "volume_access_mode": "rwm", + "volumes": [ + { + "annotations": [], + "mount_point": "/home/jovyan", + "name": "test-workspace-6lhtr", + "size": 15, + "size_type": "Gi", + "snapshot": false, + "type": "clone" + } + ] + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/Natural-Language-Processing/2. Docker/static/styles.css b/Natural-Language-Processing/2. Docker/static/styles.css index 57efc460..06916f65 100644 --- a/Natural-Language-Processing/2. Docker/static/styles.css +++ b/Natural-Language-Processing/2. Docker/static/styles.css @@ -1,90 +1,90 @@ -body { - background-color: azure; - text-align: center; - font-family: fantasy; -} - -#title { - font-size: 10vh; -} - -#center { - margin-left: 10%; - margin-right: 10%; - width: 80vw; - height: 80vh; - opacity: 2.0; - background-color: blanchedalmond; - border: 5px solid #666; - border-radius: 50px; - box-shadow:-9px 12px 9px black -} - -#input { - float: left; - width: 50%; - height: 100%; -} - -#output { - float: right; - width: 50%; - height: 100%; -} - -.content-title { - font-size: 5vh; - margin: 2%; -} - -textarea { - margin: 5%; - width: 90%; - font-size: 4vh; -} - -input { - width: 30%; - height: 10%; - font-size: 4vh; - font-family: fantasy; - background-color: lawngreen; - border-radius: 50px; -} - -.content-setting { - font-size: 4vh; - text-align: left; - margin-left: 5%; -} - -#preprocess { - border: 5px solid grey; - border-radius: 10px; - width: 87%; - height: 25%; - margin: 5%; - background-color: white; - font-size: 4vh; - overflow: auto; - word-break: break-all; - word-wrap: break-word; - text-align: left; - font-family: monospace; -} - -table { - border: 5px solid grey; - border-radius: 10px; - font-size: 4vh; - margin: 5%; - width: 90%; - height: 25%; - background-color: white; -} - -.predict-title { - width: 33.33%; - height: 20%; - color: green; +body { + background-color: azure; + text-align: center; + font-family: fantasy; +} + +#title { + font-size: 10vh; +} + +#center { + margin-left: 10%; + margin-right: 10%; + width: 80vw; + height: 80vh; + opacity: 2.0; + background-color: blanchedalmond; + border: 5px solid #666; + border-radius: 50px; + box-shadow:-9px 12px 9px black +} + +#input { + float: left; + width: 50%; + height: 100%; +} + +#output { + float: right; + width: 50%; + height: 100%; +} + +.content-title { + font-size: 5vh; + margin: 2%; +} + +textarea { + margin: 5%; + width: 90%; + font-size: 4vh; +} + +input { + width: 30%; + height: 10%; + font-size: 4vh; + font-family: fantasy; + background-color: lawngreen; + border-radius: 50px; +} + +.content-setting { + font-size: 4vh; + text-align: left; + margin-left: 5%; +} + +#preprocess { + border: 5px solid grey; + border-radius: 10px; + width: 87%; + height: 25%; + margin: 5%; + background-color: white; + font-size: 4vh; + overflow: auto; + word-break: break-all; + word-wrap: break-word; + text-align: left; + font-family: monospace; +} + +table { + border: 5px solid grey; + border-radius: 10px; + font-size: 4vh; + margin: 5%; + width: 90%; + height: 25%; + background-color: white; +} + +.predict-title { + width: 33.33%; + height: 20%; + color: green; } \ No newline at end of file diff --git a/Natural-Language-Processing/2. Docker/templates/home.html b/Natural-Language-Processing/2. Docker/templates/home.html index 9f591e96..77bd2046 100644 --- a/Natural-Language-Processing/2. Docker/templates/home.html +++ b/Natural-Language-Processing/2. Docker/templates/home.html @@ -1,156 +1,156 @@ - - - - Kubeflow - NLP - - -
Kubeflow - NLP
-
-
-
Message
-
- - -
-
-
-
Result
-
Positive +
-
Negative -
-
- {% print(data) %} -
- - - - - - - - - - - - - -
NumpySKlearnPytorchSVM
- {% if my_prediction_np == 1%} -
+
- {% elif my_prediction_np == 0%} -
-
- {% endif %} - 		- {% if my_prediction_skl == 1%} -
+
- {% elif my_prediction_skl == 0%} -
-
- {% endif %} - 		- {% if my_prediction_toc == 1%} -
+
- {% elif my_prediction_toc == 0%} -
-
- {% endif %} - 		- {% if my_prediction_svm == 1%} -
+
- {% elif my_prediction_svm == 0%} -
-
- {% endif %} -
-
-
- - - + + + + Kubeflow - NLP + + +
Kubeflow - NLP
+
+
+
Message
+
+ + +
+
+
+
Result
+
Positive +
+
Negative -
+
+ {% print(data) %} +
+ + + + + + + + + + + + + +
NumpySKlearnPytorchSVM
+ {% if my_prediction_np == 1%} +
+
+ {% elif my_prediction_np == 0%} +
-
+ {% endif %} + 		+ {% if my_prediction_skl == 1%} +
+
+ {% elif my_prediction_skl == 0%} +
-
+ {% endif %} + 		+ {% if my_prediction_toc == 1%} +
+
+ {% elif my_prediction_toc == 0%} +
-
+ {% endif %} + 		+ {% if my_prediction_svm == 1%} +
+
+ {% elif my_prediction_svm == 0%} +
-
+ {% endif %} +
+
+
+ + + diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/FL_pipeline.png b/RAG-pipeline-with-nodered/RAG_with_node-red/FL_pipeline.png new file mode 100644 index 00000000..daab79b4 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/FL_pipeline.png differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/build_flow.png b/RAG-pipeline-with-nodered/RAG_with_node-red/build_flow.png new file mode 100644 index 00000000..5708f30c Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/build_flow.png differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/edit_node.png b/RAG-pipeline-with-nodered/RAG_with_node-red/edit_node.png new file mode 100644 index 00000000..9d0b6dee Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/edit_node.png differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/Dockerfile b/RAG-pipeline-with-nodered/RAG_with_node-red/example/Dockerfile new file mode 100644 index 00000000..20d5365a --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/Dockerfile @@ -0,0 +1,24 @@ +FROM nodered/node-red:2.2.3-12 + +ARG PREFIX + +ENV NODE_OPTIONS=--max_old_space_size=128 + +USER root +RUN apk update && \ + apk add py3-pip python3-dev libffi-dev + +USER root +RUN apk update && \ + apk add py3-pip +RUN apk update && apk add python3-dev +USER node-red + +# Copy package.json to the WORKDIR so npm builds all +# of your added nodes modules for Node-RED +COPY package.json . +RUN npm install --unsafe-perm --no-update-notifier --no-fund --only=production + +ADD scripts/entrypoint.sh . + +ENTRYPOINT ["./entrypoint.sh"] \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/build.sh b/RAG-pipeline-with-nodered/RAG_with_node-red/example/build.sh new file mode 100644 index 00000000..37ad5136 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/build.sh @@ -0,0 +1,3 @@ +#!/usr/bin/env bash + +docker compose build diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/buckets/.bloomcycle.bin/xl.meta b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/buckets/.bloomcycle.bin/xl.meta new file mode 100644 index 00000000..7cbd9676 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/buckets/.bloomcycle.bin/xl.meta differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/buckets/.usage.json/xl.meta b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/buckets/.usage.json/xl.meta new file mode 100644 index 00000000..04dcc8be Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/buckets/.usage.json/xl.meta differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/config/config.json/xl.meta b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/config/config.json/xl.meta new file mode 100644 index 00000000..576d707c Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/config/config.json/xl.meta differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/config/iam/format.json/xl.meta b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/config/iam/format.json/xl.meta new file mode 100644 index 00000000..3cc3eecc Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/config/iam/format.json/xl.meta differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/format.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/format.json new file mode 100644 index 00000000..134a49fc --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/format.json @@ -0,0 +1 @@ +{"version":"1","format":"xl-single","id":"fd84d37c-ca41-455e-91a5-092551ea2351","xl":{"version":"3","this":"b6e74063-d6f7-410f-ad3f-4e8fe53877e1","sets":[["b6e74063-d6f7-410f-ad3f-4e8fe53877e1"]],"distributionAlgo":"SIPMOD+PARITY"}} \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/pool.bin/xl.meta b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/pool.bin/xl.meta new file mode 100644 index 00000000..03d9f278 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/pool.bin/xl.meta differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/7a4afe59-3ebb-451e-8f3e-2aa2972ace33/xl.meta.bkp b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/7a4afe59-3ebb-451e-8f3e-2aa2972ace33/xl.meta.bkp new file mode 100644 index 00000000..232e8ac2 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/7a4afe59-3ebb-451e-8f3e-2aa2972ace33/xl.meta.bkp differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/9fb77816-7f6e-40e0-8b08-f7ca83d930d1/xl.meta.bkp b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/9fb77816-7f6e-40e0-8b08-f7ca83d930d1/xl.meta.bkp new file mode 100644 index 00000000..0cd103e2 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/9fb77816-7f6e-40e0-8b08-f7ca83d930d1/xl.meta.bkp differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/a4349609-22f1-4a1f-b126-76ba138ed9ed/xl.meta.bkp b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/a4349609-22f1-4a1f-b126-76ba138ed9ed/xl.meta.bkp new file mode 100644 index 00000000..7e89dcad Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/a4349609-22f1-4a1f-b126-76ba138ed9ed/xl.meta.bkp differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/e1bb8ce0-0f21-4822-b751-7787775fcd26/xl.meta.bkp b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/e1bb8ce0-0f21-4822-b751-7787775fcd26/xl.meta.bkp new file mode 100644 index 00000000..5cf2cd01 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/e1bb8ce0-0f21-4822-b751-7787775fcd26/xl.meta.bkp differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/eb32cdeb-d251-4717-bcd7-d215b85af238/xl.meta.bkp b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/eb32cdeb-d251-4717-bcd7-d215b85af238/xl.meta.bkp new file mode 100644 index 00000000..57a5da24 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/eb32cdeb-d251-4717-bcd7-d215b85af238/xl.meta.bkp differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/f5502d0b-3d43-4bfc-8cf8-cef3f051a433/xl.meta.bkp b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/f5502d0b-3d43-4bfc-8cf8-cef3f051a433/xl.meta.bkp new file mode 100644 index 00000000..034dbce9 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/.trash/f5502d0b-3d43-4bfc-8cf8-cef3f051a433/xl.meta.bkp differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/3539f56f-7c06-4b3f-9136-987f1f337bea b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/3539f56f-7c06-4b3f-9136-987f1f337bea new file mode 100644 index 00000000..e36ee446 Binary files /dev/null and b/RAG-pipeline-with-nodered/RAG_with_node-red/example/data/.minio.sys/tmp/3539f56f-7c06-4b3f-9136-987f1f337bea differ diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/docker-compose.yaml b/RAG-pipeline-with-nodered/RAG_with_node-red/example/docker-compose.yaml new file mode 100644 index 00000000..fdb097ae --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/docker-compose.yaml @@ -0,0 +1,43 @@ +version: '3.7' + +services: + node-red: + image: reg.footprint-ai.com/public/kube-nodered:latest + build: + context: . + dockerfile: Dockerfile + labels: + com.docker.compose.container-number: "1" + environment: + - TZ=Asia/Taipei + - USERDIR=$USERDIR + - NODE_RED_CREDENTIAL_SECRET=noderedtutorial + - KUBEFLOW_HOST=$KUBEFLOW_HOST + - KUBEFLOW_USERNAME=$KUBEFLOW_USERNAME + - KUBEFLOW_PASSWORD=$KUBEFLOW_PASSWORD + - MINIO_HOST=$MINIO_HOST + - MINIO_ROOT_USER=$MINIO_ROOT_USER + - MINIO_ROOT_PASSWORD=$MINIO_ROOT_PASSWORD + ports: + - "1880:1880" + networks: + - node-red-net + volumes: + - ./:/data + + minio: + image: minio/minio + command: server --console-address ":9001" /data + expose: + - "9000" + environment: + - MINIO_ROOT_USER=$MINIO_ROOT_USER + - MINIO_ROOT_PASSWORD=$MINIO_ROOT_PASSWORD + ports: + - "9001:9001" + - "9000:9000" + volumes: + - ./data:/data + +networks: + node-red-net: diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.nodes.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.nodes.json new file mode 100644 index 00000000..24783ce4 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.nodes.json @@ -0,0 +1,446 @@ +{ + "node-red": { + "name": "node-red", + "version": "2.2.3", + "local": false, + "user": false, + "nodes": { + "inject": { + "name": "inject", + "types": [ + "inject" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/20-inject.js" + }, + "debug": { + "name": "debug", + "types": [ + "debug" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/21-debug.js" + }, + "complete": { + "name": "complete", + "types": [ + "complete" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/24-complete.js" + }, + "catch": { + "name": "catch", + "types": [ + "catch" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/25-catch.js" + }, + "status": { + "name": "status", + "types": [ + "status" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/25-status.js" + }, + "link": { + "name": "link", + "types": [ + "link in", + "link out", + "link call" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/60-link.js" + }, + "comment": { + "name": "comment", + "types": [ + "comment" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/90-comment.js" + }, + "unknown": { + "name": "unknown", + "types": [ + "unknown" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/common/98-unknown.js" + }, + "function": { + "name": "function", + "types": [ + "function" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/10-function.js" + }, + "switch": { + "name": "switch", + "types": [ + "switch" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/10-switch.js" + }, + "change": { + "name": "change", + "types": [ + "change" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/15-change.js" + }, + "range": { + "name": "range", + "types": [ + "range" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/16-range.js" + }, + "template": { + "name": "template", + "types": [ + "template" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/80-template.js" + }, + "delay": { + "name": "delay", + "types": [ + "delay" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/89-delay.js" + }, + "trigger": { + "name": "trigger", + "types": [ + "trigger" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/89-trigger.js" + }, + "exec": { + "name": "exec", + "types": [ + "exec" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/90-exec.js" + }, + "rbe": { + "name": "rbe", + "types": [ + "rbe" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/function/rbe.js" + }, + "tls": { + "name": "tls", + "types": [ + "tls-config" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/05-tls.js" + }, + "httpproxy": { + "name": "httpproxy", + "types": [ + "http proxy" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/06-httpproxy.js" + }, + "mqtt": { + "name": "mqtt", + "types": [ + "mqtt in", + "mqtt out", + "mqtt-broker" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/10-mqtt.js" + }, + "httpin": { + "name": "httpin", + "types": [ + "http in", + "http response" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/21-httpin.js" + }, + "httprequest": { + "name": "httprequest", + "types": [ + "http request" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/21-httprequest.js" + }, + "websocket": { + "name": "websocket", + "types": [ + "websocket in", + "websocket out", + "websocket-listener", + "websocket-client" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/22-websocket.js" + }, + "tcpin": { + "name": "tcpin", + "types": [ + "tcp in", + "tcp out", + "tcp request" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/31-tcpin.js" + }, + "udp": { + "name": "udp", + "types": [ + "udp in", + "udp out" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/network/32-udp.js" + }, + "CSV": { + "name": "CSV", + "types": [ + "csv" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/parsers/70-CSV.js" + }, + "HTML": { + "name": "HTML", + "types": [ + "html" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/parsers/70-HTML.js" + }, + "JSON": { + "name": "JSON", + "types": [ + "json" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/parsers/70-JSON.js" + }, + "XML": { + "name": "XML", + "types": [ + "xml" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/parsers/70-XML.js" + }, + "YAML": { + "name": "YAML", + "types": [ + "yaml" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/parsers/70-YAML.js" + }, + "split": { + "name": "split", + "types": [ + "split", + "join" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/sequence/17-split.js" + }, + "sort": { + "name": "sort", + "types": [ + "sort" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/sequence/18-sort.js" + }, + "batch": { + "name": "batch", + "types": [ + "batch" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/sequence/19-batch.js" + }, + "file": { + "name": "file", + "types": [ + "file", + "file in" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/storage/10-file.js" + }, + "watch": { + "name": "watch", + "types": [ + "watch" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red", + "file": "/usr/src/node-red/node_modules/@node-red/nodes/core/storage/23-watch.js" + } + } + }, + "node-red-contrib-autoai-pipeline": { + "name": "node-red-contrib-autoai-pipeline", + "version": "0.0.1", + "local": true, + "user": false, + "nodes": { + "fl": { + "name": "fl", + "types": [ + "RAG" + ], + "enabled": true, + "local": true, + "user": false, + "module": "node-red-contrib-autoai-pipeline", + "file": "/data/main/node_modules/autoAI pipeline/RAG/RAG.js" + } + } + }, + "node-red-contrib-pythonshell": { + "name": "node-red-contrib-pythonshell", + "version": "1.5.4", + "local": false, + "user": false, + "nodes": { + "pythonshell": { + "name": "pythonshell", + "types": [ + "pythonshell in" + ], + "enabled": true, + "local": false, + "user": false, + "module": "node-red-contrib-pythonshell", + "file": "/usr/src/node-red/node_modules/node-red-contrib-pythonshell/src/pythonshell.js" + } + } + } +} \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.users.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.users.json new file mode 100644 index 00000000..8e69e209 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.users.json @@ -0,0 +1,20 @@ +{ + "_": { + "editor": { + "view": { + "view-store-zoom": false, + "view-store-position": false, + "view-show-grid": true, + "view-snap-grid": true, + "view-grid-size": 20, + "view-node-status": true, + "view-node-show-label": true, + "view-show-tips": true, + "view-show-welcome-tours": true + }, + "tours": { + "welcome": "2.2.3" + } + } + } +} \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.users.json.backup b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.users.json.backup new file mode 100644 index 00000000..c6bcd309 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.config.users.json.backup @@ -0,0 +1,17 @@ +{ + "_": { + "editor": { + "view": { + "view-store-zoom": false, + "view-store-position": false, + "view-show-grid": true, + "view-snap-grid": true, + "view-grid-size": 20, + "view-node-status": true, + "view-node-show-label": true, + "view-show-tips": true, + "view-show-welcome-tours": true + } + } + } +} \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.flows.json.backup b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.flows.json.backup new file mode 100644 index 00000000..f0e28d4e --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/.flows.json.backup @@ -0,0 +1,650 @@ +[ + { + "id": "f6f2187d.f17ca8", + "type": "tab", + "label": "Flow 1", + "disabled": false, + "info": "" + }, + { + "id": "c9fd2f76f856879d", + "type": "tab", + "label": "base flow", + "disabled": false, + "info": "", + "env": [] + }, + { + "id": "e0941514f4cd8be3", + "type": "tab", + "label": "流程1", + "disabled": false, + "info": "", + "env": [] + }, + { + "id": "3cc11d24.ff01a2", + "type": "comment", + "z": "f6f2187d.f17ca8", + "name": "WARNING: please check you have started this container with a volume that is mounted to /data\\n otherwise any flow changes are lost when you redeploy or upgrade the container\\n (e.g. upgrade to a more recent node-red docker image).\\n If you are using named volumes you can ignore this warning.\\n Double click or see info side panel to learn how to start Node-RED in Docker to save your work", + "info": "\nTo start docker with a bind mount volume (-v option), for example:\n\n```\ndocker run -it -p 1880:1880 -v /home/user/node_red_data:/data --name mynodered nodered/node-red\n```\n\nwhere `/home/user/node_red_data` is a directory on your host machine where you want to store your flows.\n\nIf you do not do this then you can experiment and redploy flows, but if you restart or upgrade the container the flows will be disconnected and lost. \n\nThey will still exist in a hidden data volume, which can be recovered using standard docker techniques, but that is much more complex than just starting with a named volume as described above.", + "x": 350, + "y": 80, + "wires": [] + }, + { + "id": "b126ea03f7d74573", + "type": "debug", + "z": "f6f2187d.f17ca8", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 450, + "y": 440, + "wires": [] + }, + { + "id": "692c86b087eed4a0", + "type": "pythonshell in", + "z": "f6f2187d.f17ca8", + "name": "install dependency", + "pyfile": "/data/main/py/install.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 270, + "y": 300, + "wires": [ + [ + "0b5c3b39a424dc6a" + ] + ] + }, + { + "id": "0b5c3b39a424dc6a", + "type": "debug", + "z": "f6f2187d.f17ca8", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": true, + "complete": "payload", + "targetType": "msg", + "statusVal": "payload", + "statusType": "auto", + "x": 470, + "y": 300, + "wires": [] + }, + { + "id": "3b1a0675389769a1", + "type": "comment", + "z": "f6f2187d.f17ca8", + "name": "install python dependency", + "info": "", + "x": 370, + "y": 240, + "wires": [] + }, + { + "id": "04e11dfd70107dcf", + "type": "comment", + "z": "f6f2187d.f17ca8", + "name": "read experiemnt data from kfp", + "info": "", + "x": 380, + "y": 380, + "wires": [] + }, + { + "id": "ffe395298ffe40e0", + "type": "pythonshell in", + "z": "f6f2187d.f17ca8", + "name": "list experiments", + "pyfile": "/data/main/py/api_examples/list_experiments.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 260, + "y": 440, + "wires": [ + [ + "b126ea03f7d74573" + ] + ] + }, + { + "id": "d7c0bb24de178d22", + "type": "RAG", + "z": "f6f2187d.f17ca8", + "name": "", + "url": "", + "data_name": "", + "huggingface_token": "", + "embedding_model": "", + "LLM_model": "", + "Pinecone_token": "", + "Pinecone_index": "", + "host": "", + "username": "", + "userpassword": "", + "x": 390, + "y": 580, + "wires": [ + [] + ] + }, + { + "id": "b33d174d94620977", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "false", + "statusVal": "", + "statusType": "auto", + "x": 730, + "y": 240, + "wires": [] + }, + { + "id": "6b5130cec3efaf0a", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "false", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 320, + "wires": [] + }, + { + "id": "84f40c6e98828bb5", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 770, + "y": 400, + "wires": [] + }, + { + "id": "1a03c8eab24f2d9d", + "type": "pythonshell in", + "z": "c9fd2f76f856879d", + "name": "test", + "pyfile": "/data/main/py/model35.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 510, + "y": 520, + "wires": [ + [ + "241bb407c3b1a512" + ] + ] + }, + { + "id": "241bb407c3b1a512", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 730, + "y": 520, + "wires": [] + }, + { + "id": "9211da9222209e48", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 140, + "wires": [] + }, + { + "id": "f2394d82583d42d2", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "install dependency", + "pyfile": "/data/main/py/install.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 550, + "y": 80, + "wires": [ + [ + "cec42d56788a6d86" + ] + ] + }, + { + "id": "cec42d56788a6d86", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": true, + "complete": "payload", + "targetType": "msg", + "statusVal": "payload", + "statusType": "auto", + "x": 750, + "y": 80, + "wires": [] + }, + { + "id": "6df33a3d52decf56", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "list experiments", + "pyfile": "/data/main/py/api_examples/list_experiments.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 140, + "wires": [ + [ + "9211da9222209e48" + ] + ] + }, + { + "id": "20410c4e397c7805", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "list_pipelines", + "pyfile": "/data/main/py/api_examples/list_pipelines.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 530, + "y": 200, + "wires": [ + [ + "8400360595296c8f" + ] + ] + }, + { + "id": "8400360595296c8f", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 200, + "wires": [] + }, + { + "id": "20ae5c175bd4bd4d", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "upload_pipeline", + "pyfile": "/data/main/py/api_examples/upload_pipeline.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 260, + "wires": [ + [ + "711b376a156b58a2" + ] + ] + }, + { + "id": "711b376a156b58a2", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 260, + "wires": [] + }, + { + "id": "5dc55b480c912ba0", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "create_experiment", + "pyfile": "/data/main/py/api_examples/create_experiment.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 550, + "y": 320, + "wires": [ + [ + "47ae84a2b85aec74" + ] + ] + }, + { + "id": "47ae84a2b85aec74", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 320, + "wires": [] + }, + { + "id": "bf79cd5247e6d498", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "create_run.py", + "pyfile": "/data/main/py/api_examples/create_run.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 380, + "wires": [ + [ + "41280a3e2bd16bf1" + ] + ] + }, + { + "id": "41280a3e2bd16bf1", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 380, + "wires": [] + }, + { + "id": "9c2a05f63fa71bc7", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "delete_experiment", + "pyfile": "/data/main/py/api_examples/delete_experiment.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 550, + "y": 440, + "wires": [ + [ + "5a77999e669b4c06" + ] + ] + }, + { + "id": "5a77999e669b4c06", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 440, + "wires": [] + }, + { + "id": "ef4b5e874d4c37ab", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "delete_pipeline", + "pyfile": "/data/main/py/api_examples/delete_pipeline.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 500, + "wires": [ + [ + "29e24d1ea757f2c6" + ] + ] + }, + { + "id": "29e24d1ea757f2c6", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 500, + "wires": [] + }, + { + "id": "a2c4616e7a1591b7", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "delete_run.py", + "pyfile": "/data/main/py/api_examples/delete_run.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 560, + "wires": [ + [ + "5f127d565442e111" + ] + ] + }, + { + "id": "5f127d565442e111", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 560, + "wires": [] + }, + { + "id": "80de59953d4b8556", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "get_run", + "pyfile": "/data/main/py/api_examples/get_run.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 520, + "y": 620, + "wires": [ + [ + "4ccad67787ec0207" + ] + ] + }, + { + "id": "4ccad67787ec0207", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 620, + "wires": [] + }, + { + "id": "45cde0d735c7fa70", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "kfp_login.py", + "pyfile": "/data/main/py/api_examples/kfp_login.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 530, + "y": 680, + "wires": [ + [ + "9f35716a8bed686e" + ] + ] + }, + { + "id": "9f35716a8bed686e", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 680, + "wires": [] + }, + { + "id": "b33bc4ad7e423460", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "kfp_login.py", + "pyfile": "/data/main/py/api_examples/kfp_namespace.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 530, + "y": 740, + "wires": [ + [ + "a139fa9944b79f94" + ] + ] + }, + { + "id": "a139fa9944b79f94", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 740, + "wires": [] + } +] \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/flows.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/flows.json new file mode 100644 index 00000000..84d89a70 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/flows.json @@ -0,0 +1,697 @@ +[ + { + "id": "f6f2187d.f17ca8", + "type": "tab", + "label": "Flow 1", + "disabled": false, + "info": "" + }, + { + "id": "c9fd2f76f856879d", + "type": "tab", + "label": "base flow", + "disabled": false, + "info": "", + "env": [] + }, + { + "id": "e0941514f4cd8be3", + "type": "tab", + "label": "流程1", + "disabled": false, + "info": "", + "env": [] + }, + { + "id": "3cc11d24.ff01a2", + "type": "comment", + "z": "f6f2187d.f17ca8", + "name": "WARNING: please check you have started this container with a volume that is mounted to /data\\n otherwise any flow changes are lost when you redeploy or upgrade the container\\n (e.g. upgrade to a more recent node-red docker image).\\n If you are using named volumes you can ignore this warning.\\n Double click or see info side panel to learn how to start Node-RED in Docker to save your work", + "info": "\nTo start docker with a bind mount volume (-v option), for example:\n\n```\ndocker run -it -p 1880:1880 -v /home/user/node_red_data:/data --name mynodered nodered/node-red\n```\n\nwhere `/home/user/node_red_data` is a directory on your host machine where you want to store your flows.\n\nIf you do not do this then you can experiment and redploy flows, but if you restart or upgrade the container the flows will be disconnected and lost. \n\nThey will still exist in a hidden data volume, which can be recovered using standard docker techniques, but that is much more complex than just starting with a named volume as described above.", + "x": 350, + "y": 80, + "wires": [] + }, + { + "id": "b126ea03f7d74573", + "type": "debug", + "z": "f6f2187d.f17ca8", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 450, + "y": 440, + "wires": [] + }, + { + "id": "692c86b087eed4a0", + "type": "pythonshell in", + "z": "f6f2187d.f17ca8", + "name": "install dependency", + "pyfile": "/data/main/py/install.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 270, + "y": 300, + "wires": [ + [ + "0b5c3b39a424dc6a" + ] + ] + }, + { + "id": "0b5c3b39a424dc6a", + "type": "debug", + "z": "f6f2187d.f17ca8", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": true, + "complete": "payload", + "targetType": "msg", + "statusVal": "payload", + "statusType": "auto", + "x": 470, + "y": 300, + "wires": [] + }, + { + "id": "3b1a0675389769a1", + "type": "comment", + "z": "f6f2187d.f17ca8", + "name": "install python dependency", + "info": "", + "x": 370, + "y": 240, + "wires": [] + }, + { + "id": "04e11dfd70107dcf", + "type": "comment", + "z": "f6f2187d.f17ca8", + "name": "read experiemnt data from kfp", + "info": "", + "x": 380, + "y": 380, + "wires": [] + }, + { + "id": "ffe395298ffe40e0", + "type": "pythonshell in", + "z": "f6f2187d.f17ca8", + "name": "list experiments", + "pyfile": "/data/main/py/api_examples/list_experiments.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 260, + "y": 440, + "wires": [ + [ + "b126ea03f7d74573" + ] + ] + }, + { + "id": "d7c0bb24de178d22", + "type": "RAG", + "z": "f6f2187d.f17ca8", + "name": "", + "url": "https://arxiv.org/pdf/2207.02696.pdf", + "data_name": "/yolov7.pdf", + "huggingface_token": "", + "embedding_model": "sentence-transformers/all-MiniLM-L6-v2", + "LLM_model": "openai-community/gpt2", + "Pinecone_token": "", + "Pinecone_index": "test1", + "host": "", + "username": "", + "userpassword": "", + "x": 390, + "y": 580, + "wires": [ + [ + "a4743858ec1f39eb" + ] + ] + }, + { + "id": "e659035ebcd2420d", + "type": "inject", + "z": "f6f2187d.f17ca8", + "name": "", + "props": [ + { + "p": "payload" + }, + { + "p": "topic", + "vt": "str" + } + ], + "repeat": "", + "crontab": "", + "once": false, + "onceDelay": 0.1, + "topic": "", + "payload": "", + "payloadType": "date", + "x": 250, + "y": 580, + "wires": [ + [ + "d7c0bb24de178d22" + ] + ] + }, + { + "id": "a4743858ec1f39eb", + "type": "debug", + "z": "f6f2187d.f17ca8", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "false", + "statusVal": "", + "statusType": "auto", + "x": 530, + "y": 580, + "wires": [] + }, + { + "id": "b33d174d94620977", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "false", + "statusVal": "", + "statusType": "auto", + "x": 730, + "y": 240, + "wires": [] + }, + { + "id": "6b5130cec3efaf0a", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "false", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 320, + "wires": [] + }, + { + "id": "84f40c6e98828bb5", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 770, + "y": 400, + "wires": [] + }, + { + "id": "1a03c8eab24f2d9d", + "type": "pythonshell in", + "z": "c9fd2f76f856879d", + "name": "test", + "pyfile": "/data/main/py/model35.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 510, + "y": 520, + "wires": [ + [ + "241bb407c3b1a512" + ] + ] + }, + { + "id": "241bb407c3b1a512", + "type": "debug", + "z": "c9fd2f76f856879d", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 730, + "y": 520, + "wires": [] + }, + { + "id": "9211da9222209e48", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 140, + "wires": [] + }, + { + "id": "f2394d82583d42d2", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "install dependency", + "pyfile": "/data/main/py/install.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 550, + "y": 80, + "wires": [ + [ + "cec42d56788a6d86" + ] + ] + }, + { + "id": "cec42d56788a6d86", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": true, + "complete": "payload", + "targetType": "msg", + "statusVal": "payload", + "statusType": "auto", + "x": 750, + "y": 80, + "wires": [] + }, + { + "id": "6df33a3d52decf56", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "list experiments", + "pyfile": "/data/main/py/api_examples/list_experiments.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 140, + "wires": [ + [ + "9211da9222209e48" + ] + ] + }, + { + "id": "20410c4e397c7805", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "list_pipelines", + "pyfile": "/data/main/py/api_examples/list_pipelines.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 530, + "y": 200, + "wires": [ + [ + "8400360595296c8f" + ] + ] + }, + { + "id": "8400360595296c8f", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 200, + "wires": [] + }, + { + "id": "20ae5c175bd4bd4d", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "upload_pipeline", + "pyfile": "/data/main/py/api_examples/upload_pipeline.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 260, + "wires": [ + [ + "711b376a156b58a2" + ] + ] + }, + { + "id": "711b376a156b58a2", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 260, + "wires": [] + }, + { + "id": "5dc55b480c912ba0", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "create_experiment", + "pyfile": "/data/main/py/api_examples/create_experiment.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 550, + "y": 320, + "wires": [ + [ + "47ae84a2b85aec74" + ] + ] + }, + { + "id": "47ae84a2b85aec74", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 320, + "wires": [] + }, + { + "id": "bf79cd5247e6d498", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "create_run.py", + "pyfile": "/data/main/py/api_examples/create_run.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 380, + "wires": [ + [ + "41280a3e2bd16bf1" + ] + ] + }, + { + "id": "41280a3e2bd16bf1", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 380, + "wires": [] + }, + { + "id": "9c2a05f63fa71bc7", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "delete_experiment", + "pyfile": "/data/main/py/api_examples/delete_experiment.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 550, + "y": 440, + "wires": [ + [ + "5a77999e669b4c06" + ] + ] + }, + { + "id": "5a77999e669b4c06", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 440, + "wires": [] + }, + { + "id": "ef4b5e874d4c37ab", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "delete_pipeline", + "pyfile": "/data/main/py/api_examples/delete_pipeline.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 500, + "wires": [ + [ + "29e24d1ea757f2c6" + ] + ] + }, + { + "id": "29e24d1ea757f2c6", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 500, + "wires": [] + }, + { + "id": "a2c4616e7a1591b7", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "delete_run.py", + "pyfile": "/data/main/py/api_examples/delete_run.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 540, + "y": 560, + "wires": [ + [ + "5f127d565442e111" + ] + ] + }, + { + "id": "5f127d565442e111", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 560, + "wires": [] + }, + { + "id": "80de59953d4b8556", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "get_run", + "pyfile": "/data/main/py/api_examples/get_run.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 520, + "y": 620, + "wires": [ + [ + "4ccad67787ec0207" + ] + ] + }, + { + "id": "4ccad67787ec0207", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 620, + "wires": [] + }, + { + "id": "45cde0d735c7fa70", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "kfp_login.py", + "pyfile": "/data/main/py/api_examples/kfp_login.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 530, + "y": 680, + "wires": [ + [ + "9f35716a8bed686e" + ] + ] + }, + { + "id": "9f35716a8bed686e", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 680, + "wires": [] + }, + { + "id": "b33bc4ad7e423460", + "type": "pythonshell in", + "z": "e0941514f4cd8be3", + "name": "kfp_login.py", + "pyfile": "/data/main/py/api_examples/kfp_namespace.py", + "virtualenv": "", + "continuous": true, + "stdInData": false, + "python3": true, + "x": 530, + "y": 740, + "wires": [ + [ + "a139fa9944b79f94" + ] + ] + }, + { + "id": "a139fa9944b79f94", + "type": "debug", + "z": "e0941514f4cd8be3", + "name": "", + "active": true, + "tosidebar": true, + "console": false, + "tostatus": false, + "complete": "payload", + "targetType": "msg", + "statusVal": "", + "statusType": "auto", + "x": 750, + "y": 740, + "wires": [] + } +] diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/flows_cred.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/flows_cred.json new file mode 100644 index 00000000..a674650f --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/flows_cred.json @@ -0,0 +1,3 @@ +{ + "$": "debb1e3e4666ba98dd5189a6e20b7e40jk4=" +} \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/RAG/RAG.html b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/RAG/RAG.html new file mode 100644 index 00000000..42bd9288 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/RAG/RAG.html @@ -0,0 +1,91 @@ + + + + + + + diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/RAG/RAG.js b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/RAG/RAG.js new file mode 100644 index 00000000..c94ccc93 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/RAG/RAG.js @@ -0,0 +1,120 @@ +const util = require('util'); +const snippets = require('../snippets'); +const fs = require("fs"); +const spawn = require('child_process').spawn; + +class PythonshellNode { + constructor(config) { + this.spawn = spawn; + this.onStatus = ()=>{}; + } + onInput(msg, node, out, err) { + this.pyfile = msg.path; + var spawnCmd = "python3"; + this.py = this.spawn(spawnCmd, ['-u', this.pyfile], { + cwd: this.pyfile.substring(0, this.pyfile.lastIndexOf('/')) + }); + this.onStatus({fill:"green",shape:"dot",text:"Standby"}); + + var py = this.py; + var dataString = ''; + var errString = ''; + + py.stdout.on('data', data => { + clearTimeout(this.standbyTimer) + this.onStatus({fill:"green",shape:"dot",text:"Processing data"}); + let dataStr = data.toString(); + let lines = dataStr.split('\n'); + for (let line of lines) { + if (line.startsWith("COLUMN_NAMES:")) { + msg.col_names = line.split("COLUMN_NAMES:")[1].trim(); + }else if (line.startsWith("Classnames:")){ + msg.classnames = line.split("Classnames:")[1].trim(); + }else if (line.trim() !== "") { + dataString += line + "\n"; + } + } + if (dataString != ""){ + node.warn(dataString.trim()); + } + dataString = ''; + this.standbyTimer = setTimeout(()=>{ + this.onStatus({fill:"green",shape:"dot",text:"Standby"}) + }, 2000) + }); + + py.stderr.on('data', data => { + errString += String(data); + this.onStatus({fill:"red",shape:"dot",text:"Error: " + errString}); + }); + py.stderr.on('error', console.log) + py.stdout.on('error', console.log) + py.stdin.on('error', console.log) + py.on('error', console.log) + + py.on('close', code => { + if (code) { + err('exit code: ' + code + ', ' + errString); + this.onStatus({fill:"red",shape:"dot",text:"Exited: " + code}) + } else { + out(msg); + this.onStatus({fill:"green",shape:"dot",text:"Done"}) + } + this.py = null; + setTimeout(()=>{ + this.onStatus({}) + }, 2000) + }); + }; + + onClose() { + if (this.py) { + this.py.kill(); + this.py = null; + } + } + + setStatusCallback(callback) { + this.onStatus = callback; + } +} + +module.exports = function(RED) { + function RAG(config) { + RED.nodes.createNode(this,config); + var node = this; + var pyNode = new PythonshellNode(config); + pyNode.setStatusCallback(node.status.bind(node)); + var script = ''; + node.on('input', function(msg) { + + msg.path= "/data/main/py/RAG.py" + + script = util.format(snippets.RAG, config.url, config.data_name, config.huggingface_token, config.embedding_model, + config.LLM_model, config.Pinecone_token, config.Pinecone_index, config.host, config.username, config.userpassword) + fs.writeFile(msg.path, script, function (err) { + if (err) { + node.error('Failed to write to file: ' + err.message); + return; + } + }); + + pyNode.onInput(msg, node, function(result) { + + msg.payload = config.client_num; + node.send(msg); + }, function(error) { + node.error(error); + }); + + + }); + node.on('close', ()=>pyNode.onClose()); + } + RED.nodes.registerType("RAG", RAG); + + +} + + + diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/package.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/package.json new file mode 100644 index 00000000..a09ca0bf --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/package.json @@ -0,0 +1,19 @@ +{ + "name": "node-red-contrib-autoai-pipeline", + "version": "0.0.1", + "description": "A set of Node-RED node to run pipeline on kubeflow", + "scripts": { + "test": "./node_modules/mocha/bin/mocha" + }, + "license": "ISC", + "keywords": [ "node-red" ], + "node-red": { + "nodes": { + + "fl": "RAG/RAG.js" + } + }, + "devDependencies": { + "mocha": "^5.0.4" + } +} \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/snippets.js b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/snippets.js new file mode 100644 index 00000000..0092508e --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/node_modules/autoAI pipeline/snippets.js @@ -0,0 +1,342 @@ +'use strict'; +const RAG = +` +import kfp +from kfp import dsl +from kfp.components import func_to_container_op +from typing import NamedTuple + +def RAG()-> NamedTuple('Outputs', [("output",str)]): + import torch + import requests + from langchain.text_splitter import RecursiveCharacterTextSplitter + from langchain_community.vectorstores import Pinecone + from langchain_community.document_loaders import PyPDFLoader, OnlinePDFLoader + import os + import sys + from pinecone import Pinecone + from langchain_pinecone import PineconeVectorStore + from langchain.embeddings import HuggingFaceEmbeddings + + import pinecone + from transformers import AutoTokenizer, AutoModelForCausalLM + from transformers import pipeline + from langchain import HuggingFacePipeline, PromptTemplate + from langchain.chains import RetrievalQA + from huggingface_hub import login + + + url = '%s'#Ex: "https://arxiv.org/pdf/2207.02696.pdf", a yolov7 pdf + data_name = '%s'#Ex: "/yolov7.pdf" + + huggingface_token = '%s' + embedding_model = '%s'#Ex: "sentence-transformers/all-MiniLM-L6-v2" + LLM_model = '%s'#Ex: "openai-community/gpt2" or "meta-llama/Llama-2-7b-chat-hf" + Pinecone_token = '%s' + Pinecone_index = '%s' + + + r = requests.get(url, stream=True) + with open(data_name, 'wb') as fd: + fd.write(r.content) + loader = PyPDFLoader(data_name) + data = loader.load() + + text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=20) + docs = text_splitter.split_documents(data) + + embeddings = HuggingFaceEmbeddings(model_name=embedding_model) + PINE_KEY = os.environ.get("PINECONE_API_KEY", Pinecone_token) + pc = Pinecone(api_key=Pinecone_token) + os.environ["PINECONE_API_KEY"] = Pinecone_token + index = pc.Index(Pinecone_index) + index_name = Pinecone_index + docsearch = PineconeVectorStore.from_documents(docs, embedding=embeddings, index_name=index_name) + + + login(huggingface_token) + + + tokenizer = AutoTokenizer.from_pretrained(LLM_model) + model = AutoModelForCausalLM.from_pretrained(LLM_model, + device_map='auto', + torch_dtype=torch.float16, + use_auth_token=True, + load_in_8bit=True # If you don't use GPU, comment this parameter + ) + model = pipeline("text-generation", + model=model, + tokenizer= tokenizer, + torch_dtype=torch.bfloat16, + device_map="auto", + max_new_tokens = 512, + do_sample=True, + top_k=30, + num_return_sequences=3, + eos_token_id=tokenizer.eos_token_id + ) + + llm=HuggingFacePipeline(pipeline=model, model_kwargs={'temperature':0.1}) + + SYSTEM_PROMPT = """Use the following pieces of context to answer the question at the end. + If you don't know the answer, just say that you don't know, don't try to make up an answer.""" + B_INST, E_INST = "[INST]", "[/INST]" + B_SYS, E_SYS = "<>\\n", "\\n<>\\n\\n" + + SYSTEM_PROMPT = B_SYS + SYSTEM_PROMPT + E_SYS + instruction = """ + {context} + + Question: {question} + """ + template = B_INST + SYSTEM_PROMPT + instruction + E_INST + + prompt = PromptTemplate(template=template, input_variables=["context", "question"]) + + qa_chain = RetrievalQA.from_chain_type( + llm=llm, + chain_type="stuff", + retriever=docsearch.as_retriever(search_kwargs={"k":3}), + return_source_documents=True, + chain_type_kwargs={"prompt": prompt},) + + question = questions + result = qa_chain(question) + + print(result['result']) + + output = result['result'] + return(output) + +RAG_op = func_to_container_op(RAG,packages_to_install = [ + 'requests', + 'langchain', + 'langchain-community', + 'pypdf', + 'pinecone-client', + 'langchain_pinecone', + 'sentence_transformers', + 'googletrans==3.1.0a0', + 'transformers', + 'torch', + 'bitsandbytes', + 'accelerate', + 'urllib3==1.26.15' +]) + +@dsl.pipeline() +def rag_pipeline(): + rag_task = RAG_op().set_gpu_limit(1) + +from kfp import compiler + +compiler.Compiler().compile(rag_pipeline, 'RAG_pipeline.yaml') + +import time +import kfp_server_api +import os +import requests +import string +import random +import json +from kfp import dsl +from kfp.components import func_to_container_op, OutputPath +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = "%s" +username = "%s" +password = "%s" + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +def random_suffix() : + return ''.join(random.choices(string.ascii_lowercase + string.digits, k=10)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the Experiment API class + experiment_api_instance = kfp_server_api.ExperimentServiceApi(api_client) + name="experiment-" + random_suffix() + description="This is a experiment for RAG." + resource_reference_key_id = namespaces[0] + resource_references=[kfp_server_api.models.ApiResourceReference( + key=kfp_server_api.models.ApiResourceKey( + type=kfp_server_api.models.ApiResourceType.NAMESPACE, + id=resource_reference_key_id + ), + relationship=kfp_server_api.models.ApiRelationship.OWNER + )] + body = kfp_server_api.ApiExperiment(name=name, description=description, resource_references=resource_references) # ApiExperiment | The experiment to be created. + try: + # Creates a new experiment. + experiment_api_response = experiment_api_instance.create_experiment(body) + experiment_id = experiment_api_response.id # str | The ID of the run to be retrieved. + except ApiException as e: + print("Exception when calling ExperimentServiceApi->create_experiment: %s\\n" % e) + + # Create an instance of the pipeline API class + api_instance = kfp_server_api.PipelineUploadServiceApi(api_client) + uploadfile="RAG_pipeline.yaml" + name='pipeline-' + random_suffix() + description="This is a RAG pipline." + try: + pipeline_api_response = api_instance.upload_pipeline(uploadfile, name=name, description=description) + pipeline_id = pipeline_api_response.id # str | The ID of the run to be retrieved. + except ApiException as e: + print("Exception when calling PipelineUploadServiceApi->upload_pipeline: %s\\n" % e) + + # Create an instance of the run API class + run_api_instance = kfp_server_api.RunServiceApi(api_client) + display_name = 'RAG' + random_suffix() + description = "This is a RAG run." + pipeline_spec = kfp_server_api.ApiPipelineSpec(pipeline_id=pipeline_id) + resource_reference_key_id = namespaces[0] + resource_references=[kfp_server_api.models.ApiResourceReference( + key=kfp_server_api.models.ApiResourceKey(id=experiment_id, type=kfp_server_api.models.ApiResourceType.EXPERIMENT), + relationship=kfp_server_api.models.ApiRelationship.OWNER )] + body = kfp_server_api.ApiRun(name=display_name, description=description, pipeline_spec=pipeline_spec, resource_references=resource_references) # ApiRun | + try: + # Creates a new run. + run_api_response = run_api_instance.create_run(body) + run_id = run_api_response.run.id # str | The ID of the run to be retrieved. + except ApiException as e: + print("Exception when calling RunServiceApi->create_run: %s\\n" % e) + + Completed_flag = False + polling_interval = 10 # Time in seconds between polls + + + + + while not Completed_flag: + try: + time.sleep(1) + # Finds a specific run by ID. + api_instance = run_api_instance.get_run(run_id) + output = api_instance.pipeline_runtime.workflow_manifest + output = json.loads(output) + #print(output) + + try: + nodes = output['status']['nodes'] + conditions = output['status']['conditions'] # Comfirm completion. + + except KeyError: + nodes = {} + conditions = [] + + output_value = None + Completed_flag = conditions[1]['status'] if len(conditions) > 1 else False + + ''''' + def find_all_keys(node): + if isinstance(node, dict): + for key in node.keys(): + print("Key:", key) + find_all_keys(node[key]) + elif isinstance(node, list): + for item in node: + find_all_keys(item) + + # Call the function with your JSON data + find_all_keys(output) + ''''' + + except ApiException as e: + print("Exception when calling RunServiceApi->get_run: %s\\n" % e) + break + + if not Completed_flag: + print("Pipeline is still running. Waiting...") + time.sleep(polling_interval-1) + + found_final_pvc_name = False # Add a variable to track if the PVC name has been found + + def find_final_pvc_name(node): + global found_final_pvc_name # Declare the variable as global + + if not found_final_pvc_name: # If the PVC name has not been found yet + if isinstance(node, dict): + if 'parameters' in node: + parameters = node['parameters'] + for parameter in parameters: + if 'name' in parameter and parameter['name'] == 'mypvc-name': + value = parameter.get('value') + if value and not value.startswith('{{') and not value.endswith('}}'): + found_final_pvc_name = True # Set to True after finding the PVC name + print("mypvc-name:", value) + return value + for key, value in node.items(): + result = find_final_pvc_name(value) + if result: + return result + elif isinstance(node, list): + for item in node: + result = find_final_pvc_name(item) + if result: + return result + + return None + + find_final_pvc_name(output) # Call the function to find final_pvc_name + + + found_model_func_accuracy = False + + def find_model_func_accuracy(node): + global found_model_func_accuracy # Declare the variable as global + + if not found_model_func_accuracy: # If the model-func-accuracy has not been found yet + if isinstance(node, dict): + if 'parameters' in node: + parameters = node['parameters'] + for parameter in parameters: + if 'name' in parameter and parameter['name'] == 'model-func-accuracy': + value = parameter.get('value') + if value and not value.startswith('{{') and not value.endswith('}}'): + found_model_func_accuracy = True # Set to True after finding model-func-accuracy + print("Model Accuracy:", value) + return value + + for key, value in node.items(): + result = find_model_func_accuracy(value) + if result: + return result + elif isinstance(node, list): + for item in node: + result = find_model_func_accuracy(item) + if result: + return result + + return None + + find_model_func_accuracy(output) +`; + + + + +module.exports = { + RAG +}; diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/package.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/package.json new file mode 100644 index 00000000..b788fd49 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/package.json @@ -0,0 +1,6 @@ +{ + "name": "node-red-project", + "description": "A Node-RED Project", + "version": "0.0.1", + "private": true +} \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/RAG.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/RAG.py new file mode 100644 index 00000000..e69de29b diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/create_experiment.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/create_experiment.py new file mode 100644 index 00000000..8ac8d732 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/create_experiment.py @@ -0,0 +1,59 @@ +from __future__ import print_function +import string +import random +import time +import kfp_server_api +import os +import requests +import kfp +import json +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.ExperimentServiceApi(api_client) + name="" # str | The ID of the name to be create. + description='' # str | The description experiment. + resource_reference_key_id = namespaces[0] + resource_references=[kfp_server_api.models.ApiResourceReference( + key=kfp_server_api.models.ApiResourceKey( + type=kfp_server_api.models.ApiResourceType.NAMESPACE, + id=resource_reference_key_id + ), + relationship=kfp_server_api.models.ApiRelationship.OWNER + )] + body = kfp_server_api.ApiExperiment(name=name, description=description, resource_references=resource_references) # ApiExperiment | The experiment to be created. + try: + # Creates a new experiment. + api_response = api_instance.create_experiment(body) + pprint(api_response) + except ApiException as e: + print("Exception when calling ExperimentServiceApi->create_experiment: %s\n" % e) diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/create_run.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/create_run.py new file mode 100644 index 00000000..b02691de --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/create_run.py @@ -0,0 +1,58 @@ +from __future__ import print_function +import string +import random +import time +import kfp_server_api +import os +import requests +import kfp +import json +from pprint import pprint +from kfp_server_api.rest import ApiException +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.RunServiceApi(api_client) + pipeline_id = '' # str | The ID of the pipeline. + experiment_id = '' # str | The ID of the experiment. + display_name = '' # str | The name of the run to be create. + description = '' # str | The description of run. + pipeline_spec = kfp_server_api.ApiPipelineSpec(pipeline_id=pipeline_id) + resource_reference_key_id = namespaces[0] + resource_references=[kfp_server_api.models.ApiResourceReference( + key=kfp_server_api.models.ApiResourceKey(id=experiment_id, type=kfp_server_api.models.ApiResourceType.EXPERIMENT), + relationship=kfp_server_api.models.ApiRelationship.OWNER )] + body = kfp_server_api.ApiRun(name=display_name, description=description, pipeline_spec=pipeline_spec, resource_references=resource_references) # ApiRun | + + try: + # Creates a new run. + api_response = api_instance.create_run(body) + pprint(api_response) + except ApiException as e: + print("Exception when calling RunServiceApi->create_run: %s\n" % e) \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_experiment.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_experiment.py new file mode 100644 index 00000000..31d9205c --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_experiment.py @@ -0,0 +1,51 @@ +from __future__ import print_function +import string +import random +import time +import kfp_server_api +import os +import requests +import kfp +import json +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.ExperimentServiceApi(api_client) + experiment_id = '' # str | The ID of the experiment to be deleted. + + try: + # Deletes an experiment without deleting the experiment's runs and recurring runs. + # To avoid unexpected behaviors, delete an experiment's runs and recurring runs before deleting the experiment. + api_response = api_instance.delete_experiment(experiment_id) + pprint(api_response) + except ApiException as e: + print("Exception when calling ExperimentServiceApi->delete_experiment: %s\n" % e) \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_pipeline.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_pipeline.py new file mode 100644 index 00000000..c04d595e --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_pipeline.py @@ -0,0 +1,50 @@ +from __future__ import print_function +import string +import random +import time +import kfp_server_api +import os +import requests +import kfp +import json +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.PipelineServiceApi(api_client) + id = '' # str | The ID of the pipeline to be deleted. + + try: + # Deletes a pipeline and its pipeline versions. + api_response = api_instance.delete_pipeline(id) + pprint(api_response) + except ApiException as e: + print("Exception when calling PipelineServiceApi->delete_pipeline: %s\n" % e) \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_run.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_run.py new file mode 100644 index 00000000..aefc7dca --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/delete_run.py @@ -0,0 +1,50 @@ +from __future__ import print_function +import string +import random +import time +import kfp_server_api +import os +import requests +import kfp +import json +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.RunServiceApi(api_client) + id = '' # str | The ID of the run to be deleted. + + try: + # Deletes a run. + api_response = api_instance.delete_run(id) + pprint(api_response) + except ApiException as e: + print("Exception when calling RunServiceApi->delete_run: %s\n" % e) \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/get_run.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/get_run.py new file mode 100644 index 00000000..f62c56bd --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/get_run.py @@ -0,0 +1,64 @@ +from __future__ import print_function + +import time +import kfp_server_api +import os +import requests +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces +import json + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.RunServiceApi(api_client) + run_id = '' # str | The ID of the run to be retrieved. + + try: + # Finds a specific run by ID. + api_response = api_instance.get_run(run_id) + output = api_response.pipeline_runtime.workflow_manifest + output = json.loads(output) + nodes = output['status']['nodes'] + conditions = output['status']['conditions'] # Comfirm completion. + output_value = None + + for node_id, node in nodes.items(): + if 'inputs' in node and 'parameters' in node['inputs']: + for parameter in node['inputs']['parameters']: + if parameter['name'] == 'decision-tree-classifier-Accuracy': + output_value = parameter['value'] + break + + if output_value is not None: + print(f"Decision Tree Classifier Accuracy: {output_value}") + else: + print("Parameter not found.") + except ApiException as e: + print("Exception when calling RunServiceApi->get_run: %s\n" % e) diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/kfp_login.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/kfp_login.py new file mode 100644 index 00000000..7c95abcf --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/kfp_login.py @@ -0,0 +1,93 @@ +import re +from urllib.parse import urlsplit + +import requests + +# NOTE: the following code is referred from https://github.com/kubeflow/website/issues/2916 +def get_istio_auth_session(url: str, username: str, password: str) -> dict: + """ + Determine if the specified URL is secured by Dex and try to obtain a session cookie. + WARNING: only Dex `staticPasswords` and `LDAP` authentication are currently supported + (we default default to using `staticPasswords` if both are enabled) + + :param url: Kubeflow server URL, including protocol + :param username: Dex `staticPasswords` or `LDAP` username + :param password: Dex `staticPasswords` or `LDAP` password + :return: auth session information + """ + # define the default return object + auth_session = { + "endpoint_url": url, # KF endpoint URL + "redirect_url": None, # KF redirect URL, if applicable + "dex_login_url": None, # Dex login URL (for POST of credentials) + "is_secured": None, # True if KF endpoint is secured + "session_cookie": None # Resulting session cookies in the form "key1=value1; key2=value2" + } + + # use a persistent session (for cookies) + with requests.Session() as s: + + ################ + # Determine if Endpoint is Secured + ################ + resp = s.get(url, allow_redirects=True) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {url}" + ) + + auth_session["redirect_url"] = resp.url + + # if we were NOT redirected, then the endpoint is UNSECURED + if len(resp.history) == 0: + auth_session["is_secured"] = False + return auth_session + else: + auth_session["is_secured"] = True + + ################ + # Get Dex Login URL + ################ + redirect_url_obj = urlsplit(auth_session["redirect_url"]) + + # if we are at `/auth?=xxxx` path, we need to select an auth type + if re.search(r"/auth$", redirect_url_obj.path): + # default to "staticPasswords" auth type + redirect_url_obj = redirect_url_obj._replace( + path=re.sub(r"/auth$", "/auth/local", redirect_url_obj.path) + ) + + # if we are at `/auth/xxxx/login` path, then no further action is needed (we can use it for login POST) + if re.search(r"/auth/.*/login$", redirect_url_obj.path): + auth_session["dex_login_url"] = redirect_url_obj.geturl() + + # else, we need to be redirected to the actual login page + else: + # this GET should redirect us to the `/auth/xxxx/login` path + resp = s.get(redirect_url_obj.geturl(), allow_redirects=True) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {redirect_url_obj.geturl()}" + ) + + # set the login url + auth_session["dex_login_url"] = resp.url + + ################ + # Attempt Dex Login + ################ + resp = s.post( + auth_session["dex_login_url"], + data={"login": username, "password": password}, + allow_redirects=True + ) + if len(resp.history) == 0: + raise RuntimeError( + f"Login credentials were probably invalid - " + f"No redirect after POST to: {auth_session['dex_login_url']}" + ) + + # store the session cookies in a "key1=value1; key2=value2" string + auth_session["session_cookie"] = "; ".join([f"{c.name}={c.value}" for c in s.cookies]) + + return auth_session \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/kfp_namespace.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/kfp_namespace.py new file mode 100644 index 00000000..3c7c0dfd --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/kfp_namespace.py @@ -0,0 +1,19 @@ +import os +import requests + +from typing import TypedDict + +def retrieve_namespaces(host: str, auth_session: TypedDict) -> str: + workgroup_endpoint = os.path.join(host, "api/workgroup/env-info") + + cookies = {} + cookie_tokens = auth_session["session_cookie"].split("=") + print(cookie_tokens[0]) + cookies[cookie_tokens[0]]=cookie_tokens[1] + resp = requests.get(workgroup_endpoint, cookies=cookies) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {workgroup_endpoint}" + ) + return [ns["namespace"] for ns in resp.json()["namespaces"] if + ns["role"]=="owner"] \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/list_experiments.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/list_experiments.py new file mode 100644 index 00000000..2e4729be --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/list_experiments.py @@ -0,0 +1,44 @@ +from __future__ import print_function + +import time +import kfp_server_api +import os +import requests +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +print("available namespace: {}".format(namespaces)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.ExperimentServiceApi(api_client) + resource_reference_key_type = "NAMESPACE" + resource_reference_key_id = namespaces[0] + list_experiment_response = api_instance.list_experiment(resource_reference_key_type=resource_reference_key_type, resource_reference_key_id=resource_reference_key_id) + for experiment in list_experiment_response.experiments: + pprint(experiment) diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/list_pipelines.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/list_pipelines.py new file mode 100644 index 00000000..6e2be4b5 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/list_pipelines.py @@ -0,0 +1,43 @@ +from __future__ import print_function +import string +import time +import kfp_server_api +import os +import requests +import kfp +import json +from pprint import pprint +from kfp_server_api.rest import ApiException +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + api_instance = kfp_server_api.PipelineServiceApi(api_client) + list_pipeline_response = api_instance.list_pipelines() + for pipelines in list_pipeline_response.pipelines: + print(pipelines) \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/upload_pipeline.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/upload_pipeline.py new file mode 100644 index 00000000..410cf50a --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/api_examples/upload_pipeline.py @@ -0,0 +1,52 @@ +from __future__ import print_function +import string +import random +import time +import kfp_server_api +import os +import requests +from kfp_server_api.rest import ApiException + +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + + +host = os.getenv("KUBEFLOW_HOST") +username = os.getenv("KUBEFLOW_USERNAME") +password = os.getenv("KUBEFLOW_PASSWORD") + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True + +namespaces = retrieve_namespaces(host, auth_session) +print("available namespace: {}".format(namespaces)) + +def random_suffix() -> string: + return ''.join(random.choices(string.ascii_lowercase + string.digits, k=10)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the API class + api_instance = kfp_server_api.PipelineUploadServiceApi(api_client) + uploadfile='' # The yaml file in your local path. + name='pipeline-' + random_suffix() + description='' # str | The description of pipeline. + try: + api_response = api_instance.upload_pipeline(uploadfile, name=name, description=description) + print(api_response) + except ApiException as e: + print("Exception when calling PipelineUploadServiceApi->upload_pipeline: %s\n" % e) \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/deploykserve.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/deploykserve.py new file mode 100644 index 00000000..57d06e58 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/deploykserve.py @@ -0,0 +1,205 @@ + +from __future__ import print_function + +import re +import kfp +import time +import kfp_server_api +import os +import requests +import string +import random +import json +from kfp import dsl +from kfp import components +from kfp.components import func_to_container_op, OutputPath +from kfp_server_api.rest import ApiException +from pprint import pprint +from kfp_login import get_istio_auth_session +from kfp_namespace import retrieve_namespaces + +host = "" +username = "" +password = "" + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +namespaces = str(namespaces) +namespaces = namespaces[2:-2] +#print(namespaces) + + +kfserving_op = components.load_component_from_url('https://raw.githubusercontent.com/kubeflow/pipelines/master/components/kserve/component.yaml') +@dsl.pipeline( + name='KFServing pipeline', + description='A pipeline for KFServing with PVC.' +) +def kfservingPipeline( + action='apply', + namespace=namespaces, + + pvc_name='undefined', # change pvc_name + model_name='model01'): # change model_name + + # specify the model dir located on pvc + model_pvc_uri = 'pvc://{}/{}/'.format(pvc_name, model_name) + + # create inference service resource named by model_name + isvc_yaml = ''' +apiVersion: "serving.kserve.io/v1beta1" +kind: "InferenceService" +metadata: + name: {} + namespace: {} +spec: + predictor: + sklearn: + storageUri: {} + resources: + limits: + cpu: "100m" + requests: + cpu: "100m" +'''.format(model_name, namespace, model_pvc_uri) + + + + kfserving = kfserving_op( + action=action, + inferenceservice_yaml=isvc_yaml, + ) + kfserving.set_cpu_request("500m").set_cpu_limit("500m") + + return([isvc_yaml]) + + +# Compile pipeline +kfp.compiler.Compiler().compile(kfservingPipeline, 'sklearn-kserve.yaml') + +host = "http://140.128.102.163:31740" +username = "kubeflow02@gmail.com" +password = "tkiizd" + +auth_session = get_istio_auth_session( + url=host, + username=username, + password=password + ) + +# The client must configure the authentication and authorization parameters +# in accordance with the API server security policy. +# Examples for each auth method are provided below, use the example that +# satisfies your auth use case. + +# Configure API key authorization: Bearer +configuration = kfp_server_api.Configuration( + host = os.path.join(host, "pipeline"), +) +configuration.debug = True +namespaces = retrieve_namespaces(host, auth_session) +#print("available namespace: {}".format(namespaces)) + +def random_suffix() : + return ''.join(random.choices(string.ascii_lowercase + string.digits, k=10)) + +# Enter a context with an instance of the API client +with kfp_server_api.ApiClient(configuration, cookie=auth_session["session_cookie"]) as api_client: + # Create an instance of the Experiment API class + experiment_api_instance = kfp_server_api.ExperimentServiceApi(api_client) + name="experiment-" + random_suffix() + description="This is a experiment for only_kfserving." + resource_reference_key_id = namespaces[0] + resource_references=[kfp_server_api.models.ApiResourceReference( + key=kfp_server_api.models.ApiResourceKey( + type=kfp_server_api.models.ApiResourceType.NAMESPACE, + id=resource_reference_key_id + ), + relationship=kfp_server_api.models.ApiRelationship.OWNER + )] + body = kfp_server_api.ApiExperiment(name=name, description=description, resource_references=resource_references) # ApiExperiment | The experiment to be created. + try: + # Creates a new experiment. + experiment_api_response = experiment_api_instance.create_experiment(body) + experiment_id = experiment_api_response.id # str | The ID of the run to be retrieved. + except ApiException as e: + print("Exception when calling ExperimentServiceApi->create_experiment: %s\n" % e) + + # Create an instance of the pipeline API class + api_instance = kfp_server_api.PipelineUploadServiceApi(api_client) + uploadfile='sklearn-kserve.yaml' + name='pipeline-' + random_suffix() + description="This is a only_kfserving pipline." + try: + pipeline_api_response = api_instance.upload_pipeline(uploadfile, name=name, description=description) + pipeline_id = pipeline_api_response.id # str | The ID of the run to be retrieved. + except ApiException as e: + print("Exception when calling PipelineUploadServiceApi->upload_pipeline: %s\n" % e) + + # Create an instance of the run API class + run_api_instance = kfp_server_api.RunServiceApi(api_client) + display_name = 'run_only_kfserving' + random_suffix() + description = "This is a only_kfserving run." + pipeline_spec = kfp_server_api.ApiPipelineSpec(pipeline_id=pipeline_id) + resource_reference_key_id = namespaces[0] + resource_references=[kfp_server_api.models.ApiResourceReference( + key=kfp_server_api.models.ApiResourceKey(id=experiment_id, type=kfp_server_api.models.ApiResourceType.EXPERIMENT), + relationship=kfp_server_api.models.ApiRelationship.OWNER )] + body = kfp_server_api.ApiRun(name=display_name, description=description, pipeline_spec=pipeline_spec, resource_references=resource_references) # ApiRun | + try: + # Creates a new run. + run_api_response = run_api_instance.create_run(body) + run_id = run_api_response.run.id # str | The ID of the run to be retrieved. + except ApiException as e: + print("Exception when calling RunServiceApi->create_run: %s\n" % e) + + Completed_flag = False + polling_interval = 10 # Time in seconds between polls + + while not Completed_flag: + try: + time.sleep(1) + # Finds a specific run by ID. + api_instance = run_api_instance.get_run(run_id) + output = api_instance.pipeline_runtime.workflow_manifest + output = json.loads(output) + + try: + nodes = output['status']['nodes'] + conditions = output['status']['conditions'] # Confirm completion. + + except KeyError: + nodes = {} + conditions = [] + + Completed_flag = conditions[1]['status'] if len(conditions) > 1 else False + + except ApiException as e: + print("Exception when calling RunServiceApi->get_run: %s\n" % e) + break + + if not Completed_flag: + print("Pipeline is still running. Waiting...") + time.sleep(polling_interval-1) + else: + print("Successful Deployment!") + + + + diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/examples/kfp_login.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/examples/kfp_login.py new file mode 100644 index 00000000..7c95abcf --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/examples/kfp_login.py @@ -0,0 +1,93 @@ +import re +from urllib.parse import urlsplit + +import requests + +# NOTE: the following code is referred from https://github.com/kubeflow/website/issues/2916 +def get_istio_auth_session(url: str, username: str, password: str) -> dict: + """ + Determine if the specified URL is secured by Dex and try to obtain a session cookie. + WARNING: only Dex `staticPasswords` and `LDAP` authentication are currently supported + (we default default to using `staticPasswords` if both are enabled) + + :param url: Kubeflow server URL, including protocol + :param username: Dex `staticPasswords` or `LDAP` username + :param password: Dex `staticPasswords` or `LDAP` password + :return: auth session information + """ + # define the default return object + auth_session = { + "endpoint_url": url, # KF endpoint URL + "redirect_url": None, # KF redirect URL, if applicable + "dex_login_url": None, # Dex login URL (for POST of credentials) + "is_secured": None, # True if KF endpoint is secured + "session_cookie": None # Resulting session cookies in the form "key1=value1; key2=value2" + } + + # use a persistent session (for cookies) + with requests.Session() as s: + + ################ + # Determine if Endpoint is Secured + ################ + resp = s.get(url, allow_redirects=True) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {url}" + ) + + auth_session["redirect_url"] = resp.url + + # if we were NOT redirected, then the endpoint is UNSECURED + if len(resp.history) == 0: + auth_session["is_secured"] = False + return auth_session + else: + auth_session["is_secured"] = True + + ################ + # Get Dex Login URL + ################ + redirect_url_obj = urlsplit(auth_session["redirect_url"]) + + # if we are at `/auth?=xxxx` path, we need to select an auth type + if re.search(r"/auth$", redirect_url_obj.path): + # default to "staticPasswords" auth type + redirect_url_obj = redirect_url_obj._replace( + path=re.sub(r"/auth$", "/auth/local", redirect_url_obj.path) + ) + + # if we are at `/auth/xxxx/login` path, then no further action is needed (we can use it for login POST) + if re.search(r"/auth/.*/login$", redirect_url_obj.path): + auth_session["dex_login_url"] = redirect_url_obj.geturl() + + # else, we need to be redirected to the actual login page + else: + # this GET should redirect us to the `/auth/xxxx/login` path + resp = s.get(redirect_url_obj.geturl(), allow_redirects=True) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {redirect_url_obj.geturl()}" + ) + + # set the login url + auth_session["dex_login_url"] = resp.url + + ################ + # Attempt Dex Login + ################ + resp = s.post( + auth_session["dex_login_url"], + data={"login": username, "password": password}, + allow_redirects=True + ) + if len(resp.history) == 0: + raise RuntimeError( + f"Login credentials were probably invalid - " + f"No redirect after POST to: {auth_session['dex_login_url']}" + ) + + # store the session cookies in a "key1=value1; key2=value2" string + auth_session["session_cookie"] = "; ".join([f"{c.name}={c.value}" for c in s.cookies]) + + return auth_session \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/examples/kfp_namespace.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/examples/kfp_namespace.py new file mode 100644 index 00000000..3c7c0dfd --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/examples/kfp_namespace.py @@ -0,0 +1,19 @@ +import os +import requests + +from typing import TypedDict + +def retrieve_namespaces(host: str, auth_session: TypedDict) -> str: + workgroup_endpoint = os.path.join(host, "api/workgroup/env-info") + + cookies = {} + cookie_tokens = auth_session["session_cookie"].split("=") + print(cookie_tokens[0]) + cookies[cookie_tokens[0]]=cookie_tokens[1] + resp = requests.get(workgroup_endpoint, cookies=cookies) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {workgroup_endpoint}" + ) + return [ns["namespace"] for ns in resp.json()["namespaces"] if + ns["role"]=="owner"] \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/install.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/install.py new file mode 100644 index 00000000..75d0eebf --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/install.py @@ -0,0 +1,27 @@ +import os +import subprocess + +def python3_version(): + return subprocess.check_call(["python3", "--version"]) + +def which(command): + return subprocess.check_call(["which", command]) + +def pip3_install_requirements(): + return subprocess.check_call(["pip3", "install", "-r", "requirements.txt", + "--user"]) + +def pip3_install_kfp_sever_api(): + return subprocess.check_call(["pip3", "install", + "git+https://github.com/kubeflow/pipelines.git@1.8.19#subdirectory=backend/api/python_http_client", + "--user"]) + +def pip3_install_kfp(): + return subprocess.check_call(["pip3", "install", "kfp==1.7", "--user"]) + +python3_version() +pip3_install_requirements() +#pip3_install_kfp_sever_api() +pip3_install_kfp() + +print("done") diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/kfp_login.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/kfp_login.py new file mode 100644 index 00000000..7c95abcf --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/kfp_login.py @@ -0,0 +1,93 @@ +import re +from urllib.parse import urlsplit + +import requests + +# NOTE: the following code is referred from https://github.com/kubeflow/website/issues/2916 +def get_istio_auth_session(url: str, username: str, password: str) -> dict: + """ + Determine if the specified URL is secured by Dex and try to obtain a session cookie. + WARNING: only Dex `staticPasswords` and `LDAP` authentication are currently supported + (we default default to using `staticPasswords` if both are enabled) + + :param url: Kubeflow server URL, including protocol + :param username: Dex `staticPasswords` or `LDAP` username + :param password: Dex `staticPasswords` or `LDAP` password + :return: auth session information + """ + # define the default return object + auth_session = { + "endpoint_url": url, # KF endpoint URL + "redirect_url": None, # KF redirect URL, if applicable + "dex_login_url": None, # Dex login URL (for POST of credentials) + "is_secured": None, # True if KF endpoint is secured + "session_cookie": None # Resulting session cookies in the form "key1=value1; key2=value2" + } + + # use a persistent session (for cookies) + with requests.Session() as s: + + ################ + # Determine if Endpoint is Secured + ################ + resp = s.get(url, allow_redirects=True) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {url}" + ) + + auth_session["redirect_url"] = resp.url + + # if we were NOT redirected, then the endpoint is UNSECURED + if len(resp.history) == 0: + auth_session["is_secured"] = False + return auth_session + else: + auth_session["is_secured"] = True + + ################ + # Get Dex Login URL + ################ + redirect_url_obj = urlsplit(auth_session["redirect_url"]) + + # if we are at `/auth?=xxxx` path, we need to select an auth type + if re.search(r"/auth$", redirect_url_obj.path): + # default to "staticPasswords" auth type + redirect_url_obj = redirect_url_obj._replace( + path=re.sub(r"/auth$", "/auth/local", redirect_url_obj.path) + ) + + # if we are at `/auth/xxxx/login` path, then no further action is needed (we can use it for login POST) + if re.search(r"/auth/.*/login$", redirect_url_obj.path): + auth_session["dex_login_url"] = redirect_url_obj.geturl() + + # else, we need to be redirected to the actual login page + else: + # this GET should redirect us to the `/auth/xxxx/login` path + resp = s.get(redirect_url_obj.geturl(), allow_redirects=True) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {redirect_url_obj.geturl()}" + ) + + # set the login url + auth_session["dex_login_url"] = resp.url + + ################ + # Attempt Dex Login + ################ + resp = s.post( + auth_session["dex_login_url"], + data={"login": username, "password": password}, + allow_redirects=True + ) + if len(resp.history) == 0: + raise RuntimeError( + f"Login credentials were probably invalid - " + f"No redirect after POST to: {auth_session['dex_login_url']}" + ) + + # store the session cookies in a "key1=value1; key2=value2" string + auth_session["session_cookie"] = "; ".join([f"{c.name}={c.value}" for c in s.cookies]) + + return auth_session \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/kfp_namespace.py b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/kfp_namespace.py new file mode 100644 index 00000000..3c7c0dfd --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/kfp_namespace.py @@ -0,0 +1,19 @@ +import os +import requests + +from typing import TypedDict + +def retrieve_namespaces(host: str, auth_session: TypedDict) -> str: + workgroup_endpoint = os.path.join(host, "api/workgroup/env-info") + + cookies = {} + cookie_tokens = auth_session["session_cookie"].split("=") + print(cookie_tokens[0]) + cookies[cookie_tokens[0]]=cookie_tokens[1] + resp = requests.get(workgroup_endpoint, cookies=cookies) + if resp.status_code != 200: + raise RuntimeError( + f"HTTP status code '{resp.status_code}' for GET against: {workgroup_endpoint}" + ) + return [ns["namespace"] for ns in resp.json()["namespaces"] if + ns["role"]=="owner"] \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_decision_tree.yaml b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_decision_tree.yaml new file mode 100644 index 00000000..b1a4a1e2 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_decision_tree.yaml @@ -0,0 +1,131 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Workflow +metadata: + generateName: only-decision-tree- + annotations: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.20, pipelines.kubeflow.org/pipeline_compilation_time: '2023-04-28T21:42:00.682134', + pipelines.kubeflow.org/pipeline_spec: '{"description": "Applies Decision Tree + for classification problem.", "name": "only_decision_tree"}'} + labels: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.20} +spec: + entrypoint: only-decision-tree + templates: + - name: decision-tree-classifier + container: + args: [] + command: [python, decision_tree.py, --data, /tmp/inputs/Data/data, --accuracy, + /tmp/outputs/Accuracy/data] + image: lightnighttw/kubeflow:decision_tree_v2 + resources: + limits: + cpu: 2 + inputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/inputs/Data/data} + outputs: + parameters: + - name: decision-tree-classifier-Accuracy + valueFrom: {path: /tmp/outputs/Accuracy/data} + artifacts: + - {name: decision-tree-classifier-Accuracy, path: /tmp/outputs/Accuracy/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Trains + a decision tree classifier", "implementation": {"container": {"command": + ["python", "decision_tree.py", "--data", {"inputPath": "Data"}, "--accuracy", + {"outputPath": "Accuracy"}], "image": "lightnighttw/kubeflow:decision_tree_v2"}}, + "inputs": [{"description": "Path where data is stored.", "name": "Data", + "type": "LocalPath"}], "name": "Decision Tree classifier", "outputs": [{"description": + "Accuracy metric", "name": "Accuracy", "type": "Float"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "c5c232a9654213b3b222693949b71a5d561d04ed09543c20b6f8f2eab651ae0b", "url": + "decision_tree/decision_tree.yaml"}'} + - name: download-data-function + container: + args: [] + command: [python, download_data.py, --data, /tmp/outputs/Data/data] + image: lightnighttw/kubeflow:download_data + resources: + limits: + cpu: 2 + outputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/outputs/Data/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Download + toy data from sklearn datasets", "implementation": {"container": {"command": + ["python", "download_data.py", "--data", {"outputPath": "Data"}], "image": + "lightnighttw/kubeflow:download_data"}}, "name": "Download Data Function", + "outputs": [{"description": "Path where data will be stored.", "name": "Data", + "type": "LocalPath"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "467750defdccfec51c3af2a7eb853f74235f5f97329006d72bf33ff6e15ed02d", "url": + "download_data/download_data.yaml"}'} + - name: only-decision-tree + dag: + tasks: + - name: decision-tree-classifier + template: decision-tree-classifier + dependencies: [download-data-function] + arguments: + artifacts: + - {name: download-data-function-Data, from: '{{tasks.download-data-function.outputs.artifacts.download-data-function-Data}}'} + - {name: download-data-function, template: download-data-function} + - name: show-results + template: show-results + dependencies: [decision-tree-classifier] + arguments: + parameters: + - {name: decision-tree-classifier-Accuracy, value: '{{tasks.decision-tree-classifier.outputs.parameters.decision-tree-classifier-Accuracy}}'} + - name: show-results + container: + args: [--decision-tree, '{{inputs.parameters.decision-tree-classifier-Accuracy}}'] + command: + - sh + - -ec + - | + program_path=$(mktemp) + printf "%s" "$0" > "$program_path" + python3 -u "$program_path" "$@" + - | + def show_results(decision_tree): + # the results are shown. + + print(f"Decision tree (accuracy): {decision_tree}") + + import argparse + _parser = argparse.ArgumentParser(prog='Show results', description='') + _parser.add_argument("--decision-tree", dest="decision_tree", type=float, required=True, default=argparse.SUPPRESS) + _parsed_args = vars(_parser.parse_args()) + + _outputs = show_results(**_parsed_args) + image: python:3.7 + resources: + limits: + cpu: 2 + inputs: + parameters: + - {name: decision-tree-classifier-Accuracy} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"implementation": {"container": + {"args": ["--decision-tree", {"inputValue": "decision_tree"}], "command": + ["sh", "-ec", "program_path=$(mktemp)\nprintf \"%s\" \"$0\" > \"$program_path\"\npython3 + -u \"$program_path\" \"$@\"\n", "def show_results(decision_tree):\n # + the results are shown.\n\n print(f\"Decision tree (accuracy): {decision_tree}\")\n\nimport + argparse\n_parser = argparse.ArgumentParser(prog=''Show results'', description='''')\n_parser.add_argument(\"--decision-tree\", + dest=\"decision_tree\", type=float, required=True, default=argparse.SUPPRESS)\n_parsed_args + = vars(_parser.parse_args())\n\n_outputs = show_results(**_parsed_args)\n"], + "image": "python:3.7"}}, "inputs": [{"name": "decision_tree", "type": "Float"}], + "name": "Show results"}', pipelines.kubeflow.org/component_ref: '{}', pipelines.kubeflow.org/arguments.parameters: '{"decision_tree": + "{{inputs.parameters.decision-tree-classifier-Accuracy}}"}'} + arguments: + parameters: [] + serviceAccountName: pipeline-runner diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_logistic_regression.yaml b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_logistic_regression.yaml new file mode 100644 index 00000000..ea2ec748 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_logistic_regression.yaml @@ -0,0 +1,131 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Workflow +metadata: + generateName: only-logistic-regression- + annotations: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.20, pipelines.kubeflow.org/pipeline_compilation_time: '2023-04-28T21:47:07.033809', + pipelines.kubeflow.org/pipeline_spec: '{"description": "Applies Logistic Regression + for classification problem.", "name": "only_logistic_regression"}'} + labels: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.20} +spec: + entrypoint: only-logistic-regression + templates: + - name: download-data-function + container: + args: [] + command: [python, download_data.py, --data, /tmp/outputs/Data/data] + image: lightnighttw/kubeflow:download_data + resources: + limits: + cpu: 2 + outputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/outputs/Data/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Download + toy data from sklearn datasets", "implementation": {"container": {"command": + ["python", "download_data.py", "--data", {"outputPath": "Data"}], "image": + "lightnighttw/kubeflow:download_data"}}, "name": "Download Data Function", + "outputs": [{"description": "Path where data will be stored.", "name": "Data", + "type": "LocalPath"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "467750defdccfec51c3af2a7eb853f74235f5f97329006d72bf33ff6e15ed02d", "url": + "download_data/download_data.yaml"}'} + - name: logistic-regression-classifier + container: + args: [] + command: [python, logistic_regression.py, --data, /tmp/inputs/Data/data, --accuracy, + /tmp/outputs/Accuracy/data] + image: lightnighttw/kubeflow:logistic_regression + resources: + limits: + cpu: 2 + inputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/inputs/Data/data} + outputs: + parameters: + - name: logistic-regression-classifier-Accuracy + valueFrom: {path: /tmp/outputs/Accuracy/data} + artifacts: + - {name: logistic-regression-classifier-Accuracy, path: /tmp/outputs/Accuracy/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Trains + a Logistic Regression Classifier", "implementation": {"container": {"command": + ["python", "logistic_regression.py", "--data", {"inputPath": "Data"}, "--accuracy", + {"outputPath": "Accuracy"}], "image": "lightnighttw/kubeflow:logistic_regression"}}, + "inputs": [{"description": "Path where data is stored.", "name": "Data", + "type": "LocalPath"}], "name": "Logistic Regression Classifier", "outputs": + [{"description": "Accuracy metric", "name": "Accuracy", "type": "Float"}]}', + pipelines.kubeflow.org/component_ref: '{"digest": "a8d1e77d07d18a75bef200aee96f35136833fc4bb535f33fd949a307beb094c2", + "url": "logistic_regression/logistic_regression.yaml"}'} + - name: only-logistic-regression + dag: + tasks: + - {name: download-data-function, template: download-data-function} + - name: logistic-regression-classifier + template: logistic-regression-classifier + dependencies: [download-data-function] + arguments: + artifacts: + - {name: download-data-function-Data, from: '{{tasks.download-data-function.outputs.artifacts.download-data-function-Data}}'} + - name: show-results + template: show-results + dependencies: [logistic-regression-classifier] + arguments: + parameters: + - {name: logistic-regression-classifier-Accuracy, value: '{{tasks.logistic-regression-classifier.outputs.parameters.logistic-regression-classifier-Accuracy}}'} + - name: show-results + container: + args: [--logistic-regression, '{{inputs.parameters.logistic-regression-classifier-Accuracy}}'] + command: + - sh + - -ec + - | + program_path=$(mktemp) + printf "%s" "$0" > "$program_path" + python3 -u "$program_path" "$@" + - | + def show_results(logistic_regression): + # the results are shown. + + print(f"Logistic regression (accuracy): {logistic_regression}") + + import argparse + _parser = argparse.ArgumentParser(prog='Show results', description='') + _parser.add_argument("--logistic-regression", dest="logistic_regression", type=float, required=True, default=argparse.SUPPRESS) + _parsed_args = vars(_parser.parse_args()) + + _outputs = show_results(**_parsed_args) + image: python:3.7 + resources: + limits: + cpu: 2 + inputs: + parameters: + - {name: logistic-regression-classifier-Accuracy} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"implementation": {"container": + {"args": ["--logistic-regression", {"inputValue": "logistic_regression"}], + "command": ["sh", "-ec", "program_path=$(mktemp)\nprintf \"%s\" \"$0\" > + \"$program_path\"\npython3 -u \"$program_path\" \"$@\"\n", "def show_results(logistic_regression):\n # + the results are shown.\n\n print(f\"Logistic regression (accuracy): {logistic_regression}\")\n\nimport + argparse\n_parser = argparse.ArgumentParser(prog=''Show results'', description='''')\n_parser.add_argument(\"--logistic-regression\", + dest=\"logistic_regression\", type=float, required=True, default=argparse.SUPPRESS)\n_parsed_args + = vars(_parser.parse_args())\n\n_outputs = show_results(**_parsed_args)\n"], + "image": "python:3.7"}}, "inputs": [{"name": "logistic_regression", "type": + "Float"}], "name": "Show results"}', pipelines.kubeflow.org/component_ref: '{}', + pipelines.kubeflow.org/arguments.parameters: '{"logistic_regression": "{{inputs.parameters.logistic-regression-classifier-Accuracy}}"}'} + arguments: + parameters: [] + serviceAccountName: pipeline-runner diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_randomforest.yaml b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_randomforest.yaml new file mode 100644 index 00000000..e7dca235 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/only_randomforest.yaml @@ -0,0 +1,131 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Workflow +metadata: + generateName: three-pipeline- + annotations: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.20, pipelines.kubeflow.org/pipeline_compilation_time: '2023-04-28T22:03:53.972623', + pipelines.kubeflow.org/pipeline_spec: '{"description": "Applies random forest + for classification problem.", "name": "Three Pipeline"}'} + labels: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.20} +spec: + entrypoint: three-pipeline + templates: + - name: download-data-function + container: + args: [] + command: [python, download_data.py, --data, /tmp/outputs/Data/data] + image: lightnighttw/kubeflow:download_data + resources: + limits: + cpu: 2 + outputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/outputs/Data/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Download + toy data from sklearn datasets", "implementation": {"container": {"command": + ["python", "download_data.py", "--data", {"outputPath": "Data"}], "image": + "lightnighttw/kubeflow:download_data"}}, "name": "Download Data Function", + "outputs": [{"description": "Path where data will be stored.", "name": "Data", + "type": "LocalPath"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "467750defdccfec51c3af2a7eb853f74235f5f97329006d72bf33ff6e15ed02d", "url": + "download_data/download_data.yaml"}'} + - name: random-forest-classifier + container: + args: [] + command: [python, randomforest.py, --data, /tmp/inputs/Data/data, --accuracy, + /tmp/outputs/Accuracy/data] + image: lightnighttw/kubeflow:random_forest_v4 + resources: + limits: + cpu: 2 + inputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/inputs/Data/data} + outputs: + parameters: + - name: random-forest-classifier-Accuracy + valueFrom: {path: /tmp/outputs/Accuracy/data} + artifacts: + - {name: random-forest-classifier-Accuracy, path: /tmp/outputs/Accuracy/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Trains + a random forest classifier", "implementation": {"container": {"command": + ["python", "randomforest.py", "--data", {"inputPath": "Data"}, "--accuracy", + {"outputPath": "Accuracy"}], "image": "lightnighttw/kubeflow:random_forest_v4"}}, + "inputs": [{"description": "Path where data is stored.", "name": "Data", + "type": "LocalPath"}], "name": "Random Forest classifier", "outputs": [{"description": + "Accuracy metric", "name": "Accuracy", "type": "Float"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "7b6a0d03d7954a2b4dc7ad9295871e1780c784799f719f7365b37f1a96a6f824", "url": + "randomForest/random_forest.yaml"}'} + - name: show-results + container: + args: [--random-forest, '{{inputs.parameters.random-forest-classifier-Accuracy}}'] + command: + - sh + - -ec + - | + program_path=$(mktemp) + printf "%s" "$0" > "$program_path" + python3 -u "$program_path" "$@" + - | + def show_results(random_forest): + # the results are shown. + + print(f"Random forest (accuracy): {random_forest}") + + import argparse + _parser = argparse.ArgumentParser(prog='Show results', description='') + _parser.add_argument("--random-forest", dest="random_forest", type=float, required=True, default=argparse.SUPPRESS) + _parsed_args = vars(_parser.parse_args()) + + _outputs = show_results(**_parsed_args) + image: python:3.7 + resources: + limits: + cpu: 2 + inputs: + parameters: + - {name: random-forest-classifier-Accuracy} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.20 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"implementation": {"container": + {"args": ["--random-forest", {"inputValue": "random_forest"}], "command": + ["sh", "-ec", "program_path=$(mktemp)\nprintf \"%s\" \"$0\" > \"$program_path\"\npython3 + -u \"$program_path\" \"$@\"\n", "def show_results(random_forest):\n # + the results are shown.\n\n print(f\"Random forest (accuracy): {random_forest}\")\n\nimport + argparse\n_parser = argparse.ArgumentParser(prog=''Show results'', description='''')\n_parser.add_argument(\"--random-forest\", + dest=\"random_forest\", type=float, required=True, default=argparse.SUPPRESS)\n_parsed_args + = vars(_parser.parse_args())\n\n_outputs = show_results(**_parsed_args)\n"], + "image": "python:3.7"}}, "inputs": [{"name": "random_forest", "type": "Float"}], + "name": "Show results"}', pipelines.kubeflow.org/component_ref: '{}', pipelines.kubeflow.org/arguments.parameters: '{"random_forest": + "{{inputs.parameters.random-forest-classifier-Accuracy}}"}'} + - name: three-pipeline + dag: + tasks: + - {name: download-data-function, template: download-data-function} + - name: random-forest-classifier + template: random-forest-classifier + dependencies: [download-data-function] + arguments: + artifacts: + - {name: download-data-function-Data, from: '{{tasks.download-data-function.outputs.artifacts.download-data-function-Data}}'} + - name: show-results + template: show-results + dependencies: [random-forest-classifier] + arguments: + parameters: + - {name: random-forest-classifier-Accuracy, value: '{{tasks.random-forest-classifier.outputs.parameters.random-forest-classifier-Accuracy}}'} + arguments: + parameters: [] + serviceAccountName: pipeline-runner diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/three-pipeline.yaml b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/three-pipeline.yaml new file mode 100644 index 00000000..2e10bcef --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/pipelines_yamls/three-pipeline.yaml @@ -0,0 +1,229 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Workflow +metadata: + generateName: three-pipeline- + annotations: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.9, pipelines.kubeflow.org/pipeline_compilation_time: '2023-04-28T12:05:44.365082', + pipelines.kubeflow.org/pipeline_spec: '{"description": "Applies Decision Tree, + random forest and Logistic Regression for classification problem.", "name": + "Three Pipeline"}'} + labels: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.9} +spec: + entrypoint: three-pipeline + templates: + - name: decision-tree-classifier + container: + args: [] + command: [python, decision_tree.py, --data, /tmp/inputs/Data/data, --accuracy, + /tmp/outputs/Accuracy/data] + image: lightnighttw/kubeflow:decision_tree_v2 + resources: + limits: + cpu: 2 + inputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/inputs/Data/data} + outputs: + parameters: + - name: decision-tree-classifier-Accuracy + valueFrom: {path: /tmp/outputs/Accuracy/data} + artifacts: + - {name: decision-tree-classifier-Accuracy, path: /tmp/outputs/Accuracy/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.9 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Trains + a decision tree classifier", "implementation": {"container": {"command": + ["python", "decision_tree.py", "--data", {"inputPath": "Data"}, "--accuracy", + {"outputPath": "Accuracy"}], "image": "lightnighttw/kubeflow:decision_tree_v2"}}, + "inputs": [{"description": "Path where data is stored.", "name": "Data", + "type": "LocalPath"}], "name": "Decision Tree classifier", "outputs": [{"description": + "Accuracy metric", "name": "Accuracy", "type": "Float"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "c5c232a9654213b3b222693949b71a5d561d04ed09543c20b6f8f2eab651ae0b", "url": + "decision_tree/decision_tree.yaml"}'} + - name: download-data-function + container: + args: [] + command: [python, download_data.py, --data, /tmp/outputs/Data/data] + image: lightnighttw/kubeflow:download_data + resources: + limits: + cpu: 2 + outputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/outputs/Data/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.9 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Download + toy data from sklearn datasets", "implementation": {"container": {"command": + ["python", "download_data.py", "--data", {"outputPath": "Data"}], "image": + "lightnighttw/kubeflow:download_data"}}, "name": "Download Data Function", + "outputs": [{"description": "Path where data will be stored.", "name": "Data", + "type": "LocalPath"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "467750defdccfec51c3af2a7eb853f74235f5f97329006d72bf33ff6e15ed02d", "url": + "download_data/download_data.yaml"}'} + - name: logistic-regression-classifier + container: + args: [] + command: [python, logistic_regression.py, --data, /tmp/inputs/Data/data, --accuracy, + /tmp/outputs/Accuracy/data] + image: lightnighttw/kubeflow:logistic_regression + resources: + limits: + cpu: 2 + inputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/inputs/Data/data} + outputs: + parameters: + - name: logistic-regression-classifier-Accuracy + valueFrom: {path: /tmp/outputs/Accuracy/data} + artifacts: + - {name: logistic-regression-classifier-Accuracy, path: /tmp/outputs/Accuracy/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.9 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Trains + a Logistic Regression Classifier", "implementation": {"container": {"command": + ["python", "logistic_regression.py", "--data", {"inputPath": "Data"}, "--accuracy", + {"outputPath": "Accuracy"}], "image": "lightnighttw/kubeflow:logistic_regression"}}, + "inputs": [{"description": "Path where data is stored.", "name": "Data", + "type": "LocalPath"}], "name": "Logistic Regression Classifier", "outputs": + [{"description": "Accuracy metric", "name": "Accuracy", "type": "Float"}]}', + pipelines.kubeflow.org/component_ref: '{"digest": "a8d1e77d07d18a75bef200aee96f35136833fc4bb535f33fd949a307beb094c2", + "url": "logistic_regression/logistic_regression.yaml"}'} + - name: random-forest-classifier + container: + args: [] + command: [python, randomforest.py, --data, /tmp/inputs/Data/data, --accuracy, + /tmp/outputs/Accuracy/data] + image: lightnighttw/kubeflow:random_forest_v4 + resources: + limits: + cpu: 2 + inputs: + artifacts: + - {name: download-data-function-Data, path: /tmp/inputs/Data/data} + outputs: + parameters: + - name: random-forest-classifier-Accuracy + valueFrom: {path: /tmp/outputs/Accuracy/data} + artifacts: + - {name: random-forest-classifier-Accuracy, path: /tmp/outputs/Accuracy/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.9 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Train + a random forest classifier", "implementation": {"container": {"command": + ["python", "randomforest.py", "--data", {"inputPath": "Data"}, "--accuracy", + {"outputPath": "Accuracy"}], "image": "lightnighttw/kubeflow:random_forest_v4"}}, + "inputs": [{"description": "Path where data is stored.", "name": "Data", + "type": "LocalPath"}], "name": "Random Forest classifier", "outputs": [{"description": + "Accuracy metric", "name": "Accuracy", "type": "Float"}]}', pipelines.kubeflow.org/component_ref: '{"digest": + "b49b12da3371976eddf41d662685bb49d71b419d516de65efdd90938d2c706bc", "url": + "randomForest/random_forest.yaml"}'} + - name: show-results + container: + args: [--decision-tree, '{{inputs.parameters.decision-tree-classifier-Accuracy}}', + --logistic-regression, '{{inputs.parameters.logistic-regression-classifier-Accuracy}}', + --random-forest, '{{inputs.parameters.random-forest-classifier-Accuracy}}'] + command: + - sh + - -ec + - | + program_path=$(mktemp) + printf "%s" "$0" > "$program_path" + python3 -u "$program_path" "$@" + - | + def show_results(decision_tree, logistic_regression, random_forest): + # Given the outputs from decision_tree and logistic regression components + # the results are shown. + + print(f"Decision tree (accuracy): {decision_tree}") + print(f"Logistic regression (accuracy): {logistic_regression}") + print(f"Random forest (accuracy): {random_forest}") + + import argparse + _parser = argparse.ArgumentParser(prog='Show results', description='') + _parser.add_argument("--decision-tree", dest="decision_tree", type=float, required=True, default=argparse.SUPPRESS) + _parser.add_argument("--logistic-regression", dest="logistic_regression", type=float, required=True, default=argparse.SUPPRESS) + _parser.add_argument("--random-forest", dest="random_forest", type=float, required=True, default=argparse.SUPPRESS) + _parsed_args = vars(_parser.parse_args()) + + _outputs = show_results(**_parsed_args) + image: python:3.7 + resources: + limits: + cpu: 2 + inputs: + parameters: + - {name: decision-tree-classifier-Accuracy} + - {name: logistic-regression-classifier-Accuracy} + - {name: random-forest-classifier-Accuracy} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.8.9 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"implementation": {"container": + {"args": ["--decision-tree", {"inputValue": "decision_tree"}, "--logistic-regression", + {"inputValue": "logistic_regression"}, "--random-forest", {"inputValue": + "random_forest"}], "command": ["sh", "-ec", "program_path=$(mktemp)\nprintf + \"%s\" \"$0\" > \"$program_path\"\npython3 -u \"$program_path\" \"$@\"\n", + "def show_results(decision_tree, logistic_regression, random_forest):\n # + Given the outputs from decision_tree and logistic regression components\n # + the results are shown.\n\n print(f\"Decision tree (accuracy): {decision_tree}\")\n print(f\"Logistic + regression (accuracy): {logistic_regression}\")\n print(f\"Random forest + (accuracy): {random_forest}\")\n\nimport argparse\n_parser = argparse.ArgumentParser(prog=''Show + results'', description='''')\n_parser.add_argument(\"--decision-tree\", + dest=\"decision_tree\", type=float, required=True, default=argparse.SUPPRESS)\n_parser.add_argument(\"--logistic-regression\", + dest=\"logistic_regression\", type=float, required=True, default=argparse.SUPPRESS)\n_parser.add_argument(\"--random-forest\", + dest=\"random_forest\", type=float, required=True, default=argparse.SUPPRESS)\n_parsed_args + = vars(_parser.parse_args())\n\n_outputs = show_results(**_parsed_args)\n"], + "image": "python:3.7"}}, "inputs": [{"name": "decision_tree", "type": "Float"}, + {"name": "logistic_regression", "type": "Float"}, {"name": "random_forest", + "type": "Float"}], "name": "Show results"}', pipelines.kubeflow.org/component_ref: '{}', + pipelines.kubeflow.org/arguments.parameters: '{"decision_tree": "{{inputs.parameters.decision-tree-classifier-Accuracy}}", + "logistic_regression": "{{inputs.parameters.logistic-regression-classifier-Accuracy}}", + "random_forest": "{{inputs.parameters.random-forest-classifier-Accuracy}}"}'} + - name: three-pipeline + dag: + tasks: + - name: decision-tree-classifier + template: decision-tree-classifier + dependencies: [download-data-function] + arguments: + artifacts: + - {name: download-data-function-Data, from: '{{tasks.download-data-function.outputs.artifacts.download-data-function-Data}}'} + - {name: download-data-function, template: download-data-function} + - name: logistic-regression-classifier + template: logistic-regression-classifier + dependencies: [download-data-function] + arguments: + artifacts: + - {name: download-data-function-Data, from: '{{tasks.download-data-function.outputs.artifacts.download-data-function-Data}}'} + - name: random-forest-classifier + template: random-forest-classifier + dependencies: [download-data-function] + arguments: + artifacts: + - {name: download-data-function-Data, from: '{{tasks.download-data-function.outputs.artifacts.download-data-function-Data}}'} + - name: show-results + template: show-results + dependencies: [decision-tree-classifier, logistic-regression-classifier, random-forest-classifier] + arguments: + parameters: + - {name: decision-tree-classifier-Accuracy, value: '{{tasks.decision-tree-classifier.outputs.parameters.decision-tree-classifier-Accuracy}}'} + - {name: logistic-regression-classifier-Accuracy, value: '{{tasks.logistic-regression-classifier.outputs.parameters.logistic-regression-classifier-Accuracy}}'} + - {name: random-forest-classifier-Accuracy, value: '{{tasks.random-forest-classifier.outputs.parameters.random-forest-classifier-Accuracy}}'} + arguments: + parameters: [] + serviceAccountName: pipeline-runner diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/requirements.txt b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/requirements.txt new file mode 100644 index 00000000..fe689a3c --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/requirements.txt @@ -0,0 +1,4 @@ +setuptools +wheel +PyYAML==5.3.1 +minio \ No newline at end of file diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/sklearn-kserve.yaml b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/sklearn-kserve.yaml new file mode 100644 index 00000000..853f9360 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/py/sklearn-kserve.yaml @@ -0,0 +1,184 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Workflow +metadata: + generateName: kfserving-pipeline- + annotations: {pipelines.kubeflow.org/kfp_sdk_version: 1.7.0, pipelines.kubeflow.org/pipeline_compilation_time: '2023-11-14T09:23:34.926467', + pipelines.kubeflow.org/pipeline_spec: '{"description": "A pipeline for KFServing + with PVC.", "inputs": [{"default": "apply", "name": "action", "optional": true}, + {"default": "kubeflow02", "name": "namespace", "optional": true}, {"default": + "undefined", "name": "pvc_name", "optional": true}, {"default": "model01", "name": + "model_name", "optional": true}], "name": "KFServing pipeline"}'} + labels: {pipelines.kubeflow.org/kfp_sdk_version: 1.7.0} +spec: + entrypoint: kfserving-pipeline + templates: + - name: kfserving-pipeline + inputs: + parameters: + - {name: action} + - {name: model_name} + - {name: namespace} + - {name: pvc_name} + dag: + tasks: + - name: serve-a-model-with-kserve + template: serve-a-model-with-kserve + arguments: + parameters: + - {name: action, value: '{{inputs.parameters.action}}'} + - {name: model_name, value: '{{inputs.parameters.model_name}}'} + - {name: namespace, value: '{{inputs.parameters.namespace}}'} + - {name: pvc_name, value: '{{inputs.parameters.pvc_name}}'} + - name: serve-a-model-with-kserve + container: + args: + - -u + - kservedeployer.py + - --action + - '{{inputs.parameters.action}}' + - --model-name + - '' + - --model-uri + - '' + - --canary-traffic-percent + - '100' + - --namespace + - '' + - --framework + - '' + - --runtime-version + - latest + - --resource-requests + - '{"cpu": "0.5", "memory": "512Mi"}' + - --resource-limits + - '{"cpu": "1", "memory": "1Gi"}' + - --custom-model-spec + - '{}' + - --autoscaling-target + - '0' + - --service-account + - '' + - --enable-istio-sidecar + - "True" + - --output-path + - /tmp/outputs/InferenceService_Status/data + - --inferenceservice-yaml + - |2 + + apiVersion: "serving.kserve.io/v1beta1" + kind: "InferenceService" + metadata: + name: {{inputs.parameters.model_name}} + namespace: {{inputs.parameters.namespace}} + spec: + predictor: + sklearn: + storageUri: pvc://{{inputs.parameters.pvc_name}}/{{inputs.parameters.model_name}}/ + resources: + limits: + cpu: "100m" + requests: + cpu: "100m" + - --watch-timeout + - '300' + - --min-replicas + - '-1' + - --max-replicas + - '-1' + - --request-timeout + - '60' + - --enable-isvc-status + - "True" + command: [python] + image: quay.io/aipipeline/kserve-component:v0.11.1 + resources: + limits: {cpu: 500m} + requests: {cpu: 500m} + inputs: + parameters: + - {name: action} + - {name: model_name} + - {name: namespace} + - {name: pvc_name} + outputs: + artifacts: + - {name: serve-a-model-with-kserve-InferenceService-Status, path: /tmp/outputs/InferenceService_Status/data} + metadata: + labels: + pipelines.kubeflow.org/kfp_sdk_version: 1.7.0 + pipelines.kubeflow.org/pipeline-sdk-type: kfp + pipelines.kubeflow.org/enable_caching: "true" + annotations: {pipelines.kubeflow.org/component_spec: '{"description": "Serve + Models using KServe", "implementation": {"container": {"args": ["-u", "kservedeployer.py", + "--action", {"inputValue": "Action"}, "--model-name", {"inputValue": "Model + Name"}, "--model-uri", {"inputValue": "Model URI"}, "--canary-traffic-percent", + {"inputValue": "Canary Traffic Percent"}, "--namespace", {"inputValue": + "Namespace"}, "--framework", {"inputValue": "Framework"}, "--runtime-version", + {"inputValue": "Runtime Version"}, "--resource-requests", {"inputValue": + "Resource Requests"}, "--resource-limits", {"inputValue": "Resource Limits"}, + "--custom-model-spec", {"inputValue": "Custom Model Spec"}, "--autoscaling-target", + {"inputValue": "Autoscaling Target"}, "--service-account", {"inputValue": + "Service Account"}, "--enable-istio-sidecar", {"inputValue": "Enable Istio + Sidecar"}, "--output-path", {"outputPath": "InferenceService Status"}, "--inferenceservice-yaml", + {"inputValue": "InferenceService YAML"}, "--watch-timeout", {"inputValue": + "Watch Timeout"}, "--min-replicas", {"inputValue": "Min Replicas"}, "--max-replicas", + {"inputValue": "Max Replicas"}, "--request-timeout", {"inputValue": "Request + Timeout"}, "--enable-isvc-status", {"inputValue": "Enable ISVC Status"}], + "command": ["python"], "image": "quay.io/aipipeline/kserve-component:v0.11.1"}}, + "inputs": [{"default": "create", "description": "Action to execute on KServe", + "name": "Action", "type": "String"}, {"default": "", "description": "Name + to give to the deployed model", "name": "Model Name", "type": "String"}, + {"default": "", "description": "Path of the S3 or GCS compatible directory + containing the model.", "name": "Model URI", "type": "String"}, {"default": + "100", "description": "The traffic split percentage between the candidate + model and the last ready model", "name": "Canary Traffic Percent", "type": + "String"}, {"default": "", "description": "Kubernetes namespace where the + KServe service is deployed.", "name": "Namespace", "type": "String"}, {"default": + "", "description": "Machine Learning Framework for Model Serving.", "name": + "Framework", "type": "String"}, {"default": "latest", "description": "Runtime + Version of Machine Learning Framework", "name": "Runtime Version", "type": + "String"}, {"default": "{\"cpu\": \"0.5\", \"memory\": \"512Mi\"}", "description": + "CPU and Memory requests for Model Serving", "name": "Resource Requests", + "type": "String"}, {"default": "{\"cpu\": \"1\", \"memory\": \"1Gi\"}", + "description": "CPU and Memory limits for Model Serving", "name": "Resource + Limits", "type": "String"}, {"default": "{}", "description": "Custom model + runtime container spec in JSON", "name": "Custom Model Spec", "type": "String"}, + {"default": "0", "description": "Autoscaling Target Number", "name": "Autoscaling + Target", "type": "String"}, {"default": "", "description": "ServiceAccount + to use to run the InferenceService pod", "name": "Service Account", "type": + "String"}, {"default": "True", "description": "Whether to enable istio sidecar + injection", "name": "Enable Istio Sidecar", "type": "Bool"}, {"default": + "{}", "description": "Raw InferenceService serialized YAML for deployment", + "name": "InferenceService YAML", "type": "String"}, {"default": "300", "description": + "Timeout seconds for watching until InferenceService becomes ready.", "name": + "Watch Timeout", "type": "String"}, {"default": "-1", "description": "Minimum + number of InferenceService replicas", "name": "Min Replicas", "type": "String"}, + {"default": "-1", "description": "Maximum number of InferenceService replicas", + "name": "Max Replicas", "type": "String"}, {"default": "60", "description": + "Specifies the number of seconds to wait before timing out a request to + the component.", "name": "Request Timeout", "type": "String"}, {"default": + "True", "description": "Specifies whether to store the inference service + status as the output parameter", "name": "Enable ISVC Status", "type": "Bool"}], + "name": "Serve a model with KServe", "outputs": [{"description": "Status + JSON output of InferenceService", "name": "InferenceService Status", "type": + "String"}]}', pipelines.kubeflow.org/component_ref: '{"digest": "b0379af21c170410b8b1a9606cb6cad63f99b0af53c2a5c1f0af397b53c81cd7", + "url": "https://raw.githubusercontent.com/kubeflow/pipelines/master/components/kserve/component.yaml"}', + pipelines.kubeflow.org/arguments.parameters: '{"Action": "{{inputs.parameters.action}}", + "Autoscaling Target": "0", "Canary Traffic Percent": "100", "Custom Model + Spec": "{}", "Enable ISVC Status": "True", "Enable Istio Sidecar": "True", + "Framework": "", "InferenceService YAML": "\napiVersion: \"serving.kserve.io/v1beta1\"\nkind: + \"InferenceService\"\nmetadata:\n name: {{inputs.parameters.model_name}}\n namespace: + {{inputs.parameters.namespace}}\nspec:\n predictor:\n sklearn:\n storageUri: + pvc://{{inputs.parameters.pvc_name}}/{{inputs.parameters.model_name}}/\n resources:\n limits:\n cpu: + \"100m\"\n requests:\n cpu: \"100m\"\n", "Max Replicas": + "-1", "Min Replicas": "-1", "Model Name": "", "Model URI": "", "Namespace": + "", "Request Timeout": "60", "Resource Limits": "{\"cpu\": \"1\", \"memory\": + \"1Gi\"}", "Resource Requests": "{\"cpu\": \"0.5\", \"memory\": \"512Mi\"}", + "Runtime Version": "latest", "Service Account": "", "Watch Timeout": "300"}'} + arguments: + parameters: + - {name: action, value: apply} + - {name: namespace, value: kubeflow02} + - {name: pvc_name, value: undefined} + - {name: model_name, value: model01} + serviceAccountName: pipeline-runner diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/settings.js b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/settings.js new file mode 100644 index 00000000..332013ee --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/main/settings.js @@ -0,0 +1,498 @@ +/** + * This is the default settings file provided by Node-RED. + * + * It can contain any valid JavaScript code that will get run when Node-RED + * is started. + * + * Lines that start with // are commented out. + * Each entry should be separated from the entries above and below by a comma ',' + * + * For more information about individual settings, refer to the documentation: + * https://nodered.org/docs/user-guide/runtime/configuration + * + * The settings are split into the following sections: + * - Flow File and User Directory Settings + * - Security + * - Server Settings + * - Runtime Settings + * - Editor Settings + * - Node Settings + * + **/ + +module.exports = { + +/******************************************************************************* + * Flow File and User Directory Settings + * - flowFile + * - credentialSecret + * - flowFilePretty + * - userDir + * - nodesDir + ******************************************************************************/ + + /** The file containing the flows. If not set, defaults to flows_.json **/ + flowFile: 'flows.json', + + /** By default, credentials are encrypted in storage using a generated key. To + * specify your own secret, set the following property. + * If you want to disable encryption of credentials, set this property to false. + * Note: once you set this property, do not change it - doing so will prevent + * node-red from being able to decrypt your existing credentials and they will be + * lost. + */ + //credentialSecret: "a-secret-key", + credentialSecret: process.env.NODE_RED_CREDENTIAL_SECRET, + + /** By default, the flow JSON will be formatted over multiple lines making + * it easier to compare changes when using version control. + * To disable pretty-printing of the JSON set the following property to false. + */ + flowFilePretty: true, + + /** By default, all user data is stored in a directory called `.node-red` under + * the user's home directory. To use a different location, the following + * property can be used + */ + //userDir: '/home/nol/.node-red/', + + /** Node-RED scans the `nodes` directory in the userDir to find local node files. + * The following property can be used to specify an additional directory to scan. + */ + //nodesDir: '/home/nol/.node-red/nodes', + +/******************************************************************************* + * Security + * - adminAuth + * - https + * - httpsRefreshInterval + * - requireHttps + * - httpNodeAuth + * - httpStaticAuth + ******************************************************************************/ + + /** To password protect the Node-RED editor and admin API, the following + * property can be used. See http://nodered.org/docs/security.html for details. + */ + //adminAuth: { + // type: "credentials", + // users: [{ + // username: "admin", + // password: "$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN.", + // permissions: "*" + // }] + //}, + + /** The following property can be used to enable HTTPS + * This property can be either an object, containing both a (private) key + * and a (public) certificate, or a function that returns such an object. + * See http://nodejs.org/api/https.html#https_https_createserver_options_requestlistener + * for details of its contents. + */ + + /** Option 1: static object */ + //https: { + // key: require("fs").readFileSync('privkey.pem'), + // cert: require("fs").readFileSync('cert.pem') + //}, + + /** Option 2: function that returns the HTTP configuration object */ + // https: function() { + // // This function should return the options object, or a Promise + // // that resolves to the options object + // return { + // key: require("fs").readFileSync('privkey.pem'), + // cert: require("fs").readFileSync('cert.pem') + // } + // }, + + /** If the `https` setting is a function, the following setting can be used + * to set how often, in hours, the function will be called. That can be used + * to refresh any certificates. + */ + //httpsRefreshInterval : 12, + + /** The following property can be used to cause insecure HTTP connections to + * be redirected to HTTPS. + */ + //requireHttps: true, + + /** To password protect the node-defined HTTP endpoints (httpNodeRoot), + * including node-red-dashboard, or the static content (httpStatic), the + * following properties can be used. + * The `pass` field is a bcrypt hash of the password. + * See http://nodered.org/docs/security.html#generating-the-password-hash + */ + //httpNodeAuth: {user:"user",pass:"$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN."}, + //httpStaticAuth: {user:"user",pass:"$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN."}, + +/******************************************************************************* + * Server Settings + * - uiPort + * - uiHost + * - apiMaxLength + * - httpServerOptions + * - httpAdminRoot + * - httpAdminMiddleware + * - httpNodeRoot + * - httpNodeCors + * - httpNodeMiddleware + * - httpStatic + ******************************************************************************/ + + /** the tcp port that the Node-RED web server is listening on */ + uiPort: process.env.PORT || 1880, + + /** By default, the Node-RED UI accepts connections on all IPv4 interfaces. + * To listen on all IPv6 addresses, set uiHost to "::", + * The following property can be used to listen on a specific interface. For + * example, the following would only allow connections from the local machine. + */ + //uiHost: "127.0.0.1", + + /** The maximum size of HTTP request that will be accepted by the runtime api. + * Default: 5mb + */ + //apiMaxLength: '5mb', + + /** The following property can be used to pass custom options to the Express.js + * server used by Node-RED. For a full list of available options, refer + * to http://expressjs.com/en/api.html#app.settings.table + */ + //httpServerOptions: { }, + + /** By default, the Node-RED UI is available at http://localhost:1880/ + * The following property can be used to specify a different root path. + * If set to false, this is disabled. + */ + //httpAdminRoot: '/admin', + + /** The following property can be used to add a custom middleware function + * in front of all admin http routes. For example, to set custom http + * headers. It can be a single function or an array of middleware functions. + */ + // httpAdminMiddleware: function(req,res,next) { + // // Set the X-Frame-Options header to limit where the editor + // // can be embedded + // //res.set('X-Frame-Options', 'sameorigin'); + // next(); + // }, + + + /** Some nodes, such as HTTP In, can be used to listen for incoming http requests. + * By default, these are served relative to '/'. The following property + * can be used to specifiy a different root path. If set to false, this is + * disabled. + */ + //httpNodeRoot: '/red-nodes', + + /** The following property can be used to configure cross-origin resource sharing + * in the HTTP nodes. + * See https://github.com/troygoode/node-cors#configuration-options for + * details on its contents. The following is a basic permissive set of options: + */ + //httpNodeCors: { + // origin: "*", + // methods: "GET,PUT,POST,DELETE" + //}, + + /** If you need to set an http proxy please set an environment variable + * called http_proxy (or HTTP_PROXY) outside of Node-RED in the operating system. + * For example - http_proxy=http://myproxy.com:8080 + * (Setting it here will have no effect) + * You may also specify no_proxy (or NO_PROXY) to supply a comma separated + * list of domains to not proxy, eg - no_proxy=.acme.co,.acme.co.uk + */ + + /** The following property can be used to add a custom middleware function + * in front of all http in nodes. This allows custom authentication to be + * applied to all http in nodes, or any other sort of common request processing. + * It can be a single function or an array of middleware functions. + */ + //httpNodeMiddleware: function(req,res,next) { + // // Handle/reject the request, or pass it on to the http in node by calling next(); + // // Optionally skip our rawBodyParser by setting this to true; + // //req.skipRawBodyParser = true; + // next(); + //}, + + /** When httpAdminRoot is used to move the UI to a different root path, the + * following property can be used to identify a directory of static content + * that should be served at http://localhost:1880/. + */ + //httpStatic: '/home/nol/node-red-static/', + +/******************************************************************************* + * Runtime Settings + * - lang + * - logging + * - contextStorage + * - exportGlobalContextKeys + * - externalModules + ******************************************************************************/ + + /** Uncomment the following to run node-red in your preferred language. + * Available languages include: en-US (default), ja, de, zh-CN, zh-TW, ru, ko + * Some languages are more complete than others. + */ + // lang: "de", + + /** Configure the logging output */ + logging: { + /** Only console logging is currently supported */ + console: { + /** Level of logging to be recorded. Options are: + * fatal - only those errors which make the application unusable should be recorded + * error - record errors which are deemed fatal for a particular request + fatal errors + * warn - record problems which are non fatal + errors + fatal errors + * info - record information about the general running of the application + warn + error + fatal errors + * debug - record information which is more verbose than info + info + warn + error + fatal errors + * trace - record very detailed logging + debug + info + warn + error + fatal errors + * off - turn off all logging (doesn't affect metrics or audit) + */ + level: "info", + /** Whether or not to include metric events in the log output */ + metrics: false, + /** Whether or not to include audit events in the log output */ + audit: false + } + }, + + /** Context Storage + * The following property can be used to enable context storage. The configuration + * provided here will enable file-based context that flushes to disk every 30 seconds. + * Refer to the documentation for further options: https://nodered.org/docs/api/context/ + */ + //contextStorage: { + // default: { + // module:"localfilesystem" + // }, + //}, + + /** `global.keys()` returns a list of all properties set in global context. + * This allows them to be displayed in the Context Sidebar within the editor. + * In some circumstances it is not desirable to expose them to the editor. The + * following property can be used to hide any property set in `functionGlobalContext` + * from being list by `global.keys()`. + * By default, the property is set to false to avoid accidental exposure of + * their values. Setting this to true will cause the keys to be listed. + */ + exportGlobalContextKeys: false, + + /** Configure how the runtime will handle external npm modules. + * This covers: + * - whether the editor will allow new node modules to be installed + * - whether nodes, such as the Function node are allowed to have their + * own dynamically configured dependencies. + * The allow/denyList options can be used to limit what modules the runtime + * will install/load. It can use '*' as a wildcard that matches anything. + */ + externalModules: { + // autoInstall: false, /** Whether the runtime will attempt to automatically install missing modules */ + // autoInstallRetry: 30, /** Interval, in seconds, between reinstall attempts */ + // palette: { /** Configuration for the Palette Manager */ + // allowInstall: true, /** Enable the Palette Manager in the editor */ + // allowUpdate: true, /** Allow modules to be updated in the Palette Manager */ + // allowUpload: true, /** Allow module tgz files to be uploaded and installed */ + // allowList: ['*'], + // denyList: [], + // allowUpdateList: ['*'], + // denyUpdateList: [] + // }, + // modules: { /** Configuration for node-specified modules */ + // allowInstall: true, + // allowList: [], + // denyList: [] + // } + }, + + +/******************************************************************************* + * Editor Settings + * - disableEditor + * - editorTheme + ******************************************************************************/ + + /** The following property can be used to disable the editor. The admin API + * is not affected by this option. To disable both the editor and the admin + * API, use either the httpRoot or httpAdminRoot properties + */ + //disableEditor: false, + + /** Customising the editor + * See https://nodered.org/docs/user-guide/runtime/configuration#editor-themes + * for all available options. + */ + editorTheme: { + /** The following property can be used to set a custom theme for the editor. + * See https://github.com/node-red-contrib-themes/theme-collection for + * a collection of themes to chose from. + */ + //theme: "", + + /** To disable the 'Welcome to Node-RED' tour that is displayed the first + * time you access the editor for each release of Node-RED, set this to false + */ + //tours: false, + + palette: { + /** The following property can be used to order the categories in the editor + * palette. If a node's category is not in the list, the category will get + * added to the end of the palette. + * If not set, the following default order is used: + */ + //categories: ['subflows', 'common', 'function', 'network', 'sequence', 'parser', 'storage'], + }, + + projects: { + /** To enable the Projects feature, set this value to true */ + enabled: false, + workflow: { + /** Set the default projects workflow mode. + * - manual - you must manually commit changes + * - auto - changes are automatically committed + * This can be overridden per-user from the 'Git config' + * section of 'User Settings' within the editor + */ + mode: "manual" + } + }, + + codeEditor: { + /** Select the text editor component used by the editor. + * Defaults to "ace", but can be set to "ace" or "monaco" + */ + lib: "ace", + options: { + /** The follow options only apply if the editor is set to "monaco" + * + * theme - must match the file name of a theme in + * packages/node_modules/@node-red/editor-client/src/vendor/monaco/dist/theme + * e.g. "tomorrow-night", "upstream-sunburst", "github", "my-theme" + */ + theme: "vs", + /** other overrides can be set e.g. fontSize, fontFamily, fontLigatures etc. + * for the full list, see https://microsoft.github.io/monaco-editor/api/interfaces/monaco.editor.istandaloneeditorconstructionoptions.html + */ + //fontSize: 14, + //fontFamily: "Cascadia Code, Fira Code, Consolas, 'Courier New', monospace", + //fontLigatures: true, + } + } + }, + +/******************************************************************************* + * Node Settings + * - fileWorkingDirectory + * - functionGlobalContext + * - functionExternalModules + * - nodeMessageBufferMaxLength + * - ui (for use with Node-RED Dashboard) + * - debugUseColors + * - debugMaxLength + * - execMaxBufferSize + * - httpRequestTimeout + * - mqttReconnectTime + * - serialReconnectTime + * - socketReconnectTime + * - socketTimeout + * - tcpMsgQueueSize + * - inboundWebSocketTimeout + * - tlsConfigDisableLocalFiles + * - webSocketNodeVerifyClient + ******************************************************************************/ + + /** The working directory to handle relative file paths from within the File nodes + * defaults to the working directory of the Node-RED process. + */ + //fileWorkingDirectory: "", + + /** Allow the Function node to load additional npm modules directly */ + functionExternalModules: true, + + /** The following property can be used to set predefined values in Global Context. + * This allows extra node modules to be made available with in Function node. + * For example, the following: + * functionGlobalContext: { os:require('os') } + * will allow the `os` module to be accessed in a Function node using: + * global.get("os") + */ + functionGlobalContext: { + // os:require('os'), + }, + + /** The maximum number of messages nodes will buffer internally as part of their + * operation. This applies across a range of nodes that operate on message sequences. + * defaults to no limit. A value of 0 also means no limit is applied. + */ + //nodeMessageBufferMaxLength: 0, + + /** If you installed the optional node-red-dashboard you can set it's path + * relative to httpNodeRoot + * Other optional properties include + * readOnly:{boolean}, + * middleware:{function or array}, (req,res,next) - http middleware + * ioMiddleware:{function or array}, (socket,next) - socket.io middleware + */ + //ui: { path: "ui" }, + + /** Colourise the console output of the debug node */ + //debugUseColors: true, + + /** The maximum length, in characters, of any message sent to the debug sidebar tab */ + debugMaxLength: 1000, + + /** Maximum buffer size for the exec node. Defaults to 10Mb */ + //execMaxBufferSize: 10000000, + + /** Timeout in milliseconds for HTTP request connections. Defaults to 120s */ + //httpRequestTimeout: 120000, + + /** Retry time in milliseconds for MQTT connections */ + mqttReconnectTime: 15000, + + /** Retry time in milliseconds for Serial port connections */ + serialReconnectTime: 15000, + + /** Retry time in milliseconds for TCP socket connections */ + //socketReconnectTime: 10000, + + /** Timeout in milliseconds for TCP server socket connections. Defaults to no timeout */ + //socketTimeout: 120000, + + /** Maximum number of messages to wait in queue while attempting to connect to TCP socket + * defaults to 1000 + */ + //tcpMsgQueueSize: 2000, + + /** Timeout in milliseconds for inbound WebSocket connections that do not + * match any configured node. Defaults to 5000 + */ + //inboundWebSocketTimeout: 5000, + + /** To disable the option for using local files for storing keys and + * certificates in the TLS configuration node, set this to true. + */ + //tlsConfigDisableLocalFiles: true, + + /** The following property can be used to verify websocket connection attempts. + * This allows, for example, the HTTP request headers to be checked to ensure + * they include valid authentication information. + */ + //webSocketNodeVerifyClient: function(info) { + // /** 'info' has three properties: + // * - origin : the value in the Origin header + // * - req : the HTTP request + // * - secure : true if req.connection.authorized or req.connection.encrypted is set + // * + // * The function should return true if the connection should be accepted, false otherwise. + // * + // * Alternatively, if this function is defined to accept a second argument, callback, + // * it can be used to verify the client asynchronously. + // * The callback takes three arguments: + // * - result : boolean, whether to accept the connection or not + // * - code : if result is false, the HTTP error status to return + // * - reason: if result is false, the HTTP reason string to return + // */ + //}, +} diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/package-lock.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/package-lock.json new file mode 100644 index 00000000..a333a161 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/package-lock.json @@ -0,0 +1,10782 @@ +{ + "name": "node-red", + "version": "2.2.3", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "node-red", + "version": "2.2.3", + "license": "Apache-2.0", + "dependencies": { + "acorn": "8.7.0", + "acorn-walk": "8.2.0", + "ajv": "8.10.0", + "async-mutex": "0.3.2", + "basic-auth": "2.0.1", + "bcryptjs": "2.4.3", + "body-parser": "1.19.1", + "cheerio": "1.0.0-rc.10", + "clone": "2.1.2", + "content-type": "1.0.4", + "cookie": "0.4.2", + "cookie-parser": "1.4.6", + "cors": "2.8.5", + "cronosjs": "1.7.1", + "denque": "2.0.1", + "express": "4.17.2", + "express-session": "1.17.2", + "form-data": "4.0.0", + "fs-extra": "10.0.0", + "fs.notify": "0.0.4", + "got": "11.8.3", + "hash-sum": "2.0.0", + "hpagent": "0.1.2", + "https-proxy-agent": "5.0.0", + "i18next": "21.6.11", + "iconv-lite": "0.6.3", + "is-utf8": "0.2.1", + "js-yaml": "3.14.1", + "json-stringify-safe": "5.0.1", + "jsonata": "1.8.6", + "lodash.clonedeep": "^4.5.0", + "media-typer": "1.1.0", + "memorystore": "1.6.7", + "mime": "3.0.0", + "moment-timezone": "0.5.34", + "mqtt": "4.3.5", + "multer": "1.4.4", + "mustache": "4.2.0", + "node-red": "2.2.3", + "node-red-admin": "^2.2.3", + "node-red-contrib-pythonshell": "github:namgk/node-red-contrib-pythonshell", + "nopt": "5.0.0", + "oauth2orize": "1.11.1", + "on-headers": "1.0.2", + "passport": "0.5.2", + "passport-http-bearer": "1.0.1", + "passport-oauth2-client-password": "0.1.2", + "raw-body": "2.4.3", + "semver": "7.3.5", + "tar": "6.1.11", + "tough-cookie": "4.0.0", + "uglify-js": "3.15.1", + "uuid": "8.3.2", + "ws": "7.5.6", + "xml2js": "0.4.23" + }, + "devDependencies": { + "dompurify": "2.3.6", + "grunt": "1.5.2", + "grunt-chmod": "~1.1.1", + "grunt-cli": "~1.4.3", + "grunt-concurrent": "3.0.0", + "grunt-contrib-clean": "~2.0.0", + "grunt-contrib-compress": "2.0.0", + "grunt-contrib-concat": "~1.0.1", + "grunt-contrib-copy": "~1.0.0", + "grunt-contrib-jshint": "3.1.1", + "grunt-contrib-uglify": "5.0.1", + "grunt-contrib-watch": "~1.1.0", + "grunt-jsdoc": "2.4.1", + "grunt-jsdoc-to-markdown": "6.0.0", + "grunt-jsonlint": "2.1.3", + "grunt-mkdir": "~1.1.0", + "grunt-npm-command": "~0.1.2", + "grunt-sass": "~3.1.0", + "grunt-simple-mocha": "~0.4.1", + "grunt-simple-nyc": "^3.0.1", + "i18next-http-backend": "1.3.2", + "jquery-i18next": "1.2.1", + "jsdoc-nr-template": "github:node-red/jsdoc-nr-template", + "marked": "4.0.12", + "minami": "1.2.3", + "mocha": "9.2.0", + "node-red-node-test-helper": "^0.2.7", + "nodemon": "2.0.15", + "proxy": "^1.0.2", + "sass": "1.49.7", + "should": "13.2.3", + "sinon": "11.1.2", + "stoppable": "^1.1.0", + "supertest": "6.2.2" + }, + "engines": { + "node": ">=12" + }, + "optionalDependencies": { + "bcrypt": "5.0.1" + } + }, + "node_modules/@babel/code-frame": { + "version": "7.18.6", + "resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.18.6.tgz", + "integrity": "sha512-TDCmlK5eOvH+eH7cdAFlNXeVJqWIQ7gW9tY1GJIpUtFb6CmjVyq2VM3u71bOyR8CRihcCgMUYoDNyLXao3+70Q==", + "dev": true, + "dependencies": { + "@babel/highlight": "^7.18.6" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/generator": { + "version": "7.21.3", + "resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.21.3.tgz", + "integrity": "sha512-QS3iR1GYC/YGUnW7IdggFeN5c1poPUurnGttOV/bZgPGV+izC/D8HnD6DLwod0fsatNyVn1G3EVWMYIF0nHbeA==", + "dev": true, + "dependencies": { + "@babel/types": "^7.21.3", + "@jridgewell/gen-mapping": "^0.3.2", + "@jridgewell/trace-mapping": "^0.3.17", + "jsesc": "^2.5.1" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-environment-visitor": { + "version": "7.18.9", + "resolved": "https://registry.npmjs.org/@babel/helper-environment-visitor/-/helper-environment-visitor-7.18.9.tgz", + "integrity": "sha512-3r/aACDJ3fhQ/EVgFy0hpj8oHyHpQc+LPtJoY9SzTThAsStm4Ptegq92vqKoE3vD706ZVFWITnMnxucw+S9Ipg==", + "dev": true, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-function-name": { + "version": "7.21.0", + "resolved": "https://registry.npmjs.org/@babel/helper-function-name/-/helper-function-name-7.21.0.tgz", + "integrity": "sha512-HfK1aMRanKHpxemaY2gqBmL04iAPOPRj7DxtNbiDOrJK+gdwkiNRVpCpUJYbUT+aZyemKN8brqTOxzCaG6ExRg==", + "dev": true, + "dependencies": { + "@babel/template": "^7.20.7", + "@babel/types": "^7.21.0" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-hoist-variables": { + "version": "7.18.6", + "resolved": "https://registry.npmjs.org/@babel/helper-hoist-variables/-/helper-hoist-variables-7.18.6.tgz", + "integrity": "sha512-UlJQPkFqFULIcyW5sbzgbkxn2FKRgwWiRexcuaR8RNJRy8+LLveqPjwZV/bwrLZCN0eUHD/x8D0heK1ozuoo6Q==", + "dev": true, + "dependencies": { + "@babel/types": "^7.18.6" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-split-export-declaration": { + "version": "7.18.6", + "resolved": "https://registry.npmjs.org/@babel/helper-split-export-declaration/-/helper-split-export-declaration-7.18.6.tgz", + "integrity": "sha512-bde1etTx6ZyTmobl9LLMMQsaizFVZrquTEHOqKeQESMKo4PlObf+8+JA25ZsIpZhT/WEd39+vOdLXAFG/nELpA==", + "dev": true, + "dependencies": { + "@babel/types": "^7.18.6" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-string-parser": { + "version": "7.19.4", + "resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.19.4.tgz", + "integrity": "sha512-nHtDoQcuqFmwYNYPz3Rah5ph2p8PFeFCsZk9A/48dPc/rGocJ5J3hAAZ7pb76VWX3fZKu+uEr/FhH5jLx7umrw==", + "dev": true, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-validator-identifier": { + "version": "7.19.1", + "resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.19.1.tgz", + "integrity": "sha512-awrNfaMtnHUr653GgGEs++LlAvW6w+DcPrOliSMXWCKo597CwL5Acf/wWdNkf/tfEQE3mjkeD1YOVZOUV/od1w==", + "dev": true, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/highlight": { + "version": "7.18.6", + "resolved": "https://registry.npmjs.org/@babel/highlight/-/highlight-7.18.6.tgz", + "integrity": "sha512-u7stbOuYjaPezCuLj29hNW1v64M2Md2qupEKP1fHc7WdOA3DgLh37suiSrZYY7haUB7iBeQZ9P1uiRF359do3g==", + "dev": true, + "dependencies": { + "@babel/helper-validator-identifier": "^7.18.6", + "chalk": "^2.0.0", + "js-tokens": "^4.0.0" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/highlight/node_modules/ansi-styles": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz", + "integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==", + "dev": true, + "dependencies": { + "color-convert": "^1.9.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/@babel/highlight/node_modules/chalk": { + "version": "2.4.2", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz", + "integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==", + "dev": true, + "dependencies": { + "ansi-styles": "^3.2.1", + "escape-string-regexp": "^1.0.5", + "supports-color": "^5.3.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/@babel/highlight/node_modules/color-convert": { + "version": "1.9.3", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.3.tgz", + "integrity": "sha512-QfAUtd+vFdAtFQcC8CCyYt1fYWxSqAiK2cSD6zDB8N3cpsEBAvRxp9zOGg6G/SHHJYAT88/az/IuDGALsNVbGg==", + "dev": true, + "dependencies": { + "color-name": "1.1.3" + } + }, + "node_modules/@babel/highlight/node_modules/color-name": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz", + "integrity": "sha512-72fSenhMw2HZMTVHeCA9KCmpEIbzWiQsjN+BHcBbS9vr1mtt+vJjPdksIBNUmKAW8TFUDPJK5SUU3QhE9NEXDw==", + "dev": true + }, + "node_modules/@babel/highlight/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/@babel/highlight/node_modules/has-flag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", + "integrity": "sha512-sKJf1+ceQBr4SMkvQnBDNDtf4TXpVhVGateu0t918bl30FnbE2m4vNLX+VWe/dpjlb+HugGYzW7uQXH98HPEYw==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/@babel/highlight/node_modules/supports-color": { + "version": "5.5.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz", + "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==", + "dev": true, + "dependencies": { + "has-flag": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/@babel/parser": { + "version": "7.21.3", + "resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.21.3.tgz", + "integrity": "sha512-lobG0d7aOfQRXh8AyklEAgZGvA4FShxo6xQbUrrT/cNBPUdIDojlokwJsQyCC/eKia7ifqM0yP+2DRZ4WKw2RQ==", + "dev": true, + "bin": { + "parser": "bin/babel-parser.js" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@babel/runtime": { + "version": "7.21.0", + "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.21.0.tgz", + "integrity": "sha512-xwII0//EObnq89Ji5AKYQaRYiW/nZ3llSv29d49IuxPhKbtJoLP+9QUUZ4nVragQVtaVGeZrpB+ZtG/Pdy/POw==", + "dependencies": { + "regenerator-runtime": "^0.13.11" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/template": { + "version": "7.20.7", + "resolved": "https://registry.npmjs.org/@babel/template/-/template-7.20.7.tgz", + "integrity": "sha512-8SegXApWe6VoNw0r9JHpSteLKTpTiLZ4rMlGIm9JQ18KiCtyQiAMEazujAHrUS5flrcqYZa75ukev3P6QmUwUw==", + "dev": true, + "dependencies": { + "@babel/code-frame": "^7.18.6", + "@babel/parser": "^7.20.7", + "@babel/types": "^7.20.7" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/traverse": { + "version": "7.21.3", + "resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.21.3.tgz", + "integrity": "sha512-XLyopNeaTancVitYZe2MlUEvgKb6YVVPXzofHgqHijCImG33b/uTurMS488ht/Hbsb2XK3U2BnSTxKVNGV3nGQ==", + "dev": true, + "dependencies": { + "@babel/code-frame": "^7.18.6", + "@babel/generator": "^7.21.3", + "@babel/helper-environment-visitor": "^7.18.9", + "@babel/helper-function-name": "^7.21.0", + "@babel/helper-hoist-variables": "^7.18.6", + "@babel/helper-split-export-declaration": "^7.18.6", + "@babel/parser": "^7.21.3", + "@babel/types": "^7.21.3", + "debug": "^4.1.0", + "globals": "^11.1.0" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/traverse/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dev": true, + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/@babel/traverse/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==", + "dev": true + }, + "node_modules/@babel/types": { + "version": "7.21.3", + "resolved": "https://registry.npmjs.org/@babel/types/-/types-7.21.3.tgz", + "integrity": "sha512-sBGdETxC+/M4o/zKC0sl6sjWv62WFR/uzxrJ6uYyMLZOUlPnwzw0tKgVHOXxaAd5l2g8pEDM5RZ495GPQI77kg==", + "dev": true, + "dependencies": { + "@babel/helper-string-parser": "^7.19.4", + "@babel/helper-validator-identifier": "^7.19.1", + "to-fast-properties": "^2.0.0" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@jridgewell/gen-mapping": { + "version": "0.3.2", + "resolved": "https://registry.npmjs.org/@jridgewell/gen-mapping/-/gen-mapping-0.3.2.tgz", + "integrity": "sha512-mh65xKQAzI6iBcFzwv28KVWSmCkdRBWoOh+bYQGW3+6OZvbbN3TqMGo5hqYxQniRcH9F2VZIoJCm4pa3BPDK/A==", + "dev": true, + "dependencies": { + "@jridgewell/set-array": "^1.0.1", + "@jridgewell/sourcemap-codec": "^1.4.10", + "@jridgewell/trace-mapping": "^0.3.9" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@jridgewell/resolve-uri": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.0.tgz", + "integrity": "sha512-F2msla3tad+Mfht5cJq7LSXcdudKTWCVYUgw6pLFOOHSTtZlj6SWNYAp+AhuqLmWdBO2X5hPrLcu8cVP8fy28w==", + "dev": true, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@jridgewell/set-array": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@jridgewell/set-array/-/set-array-1.1.2.tgz", + "integrity": "sha512-xnkseuNADM0gt2bs+BvhO0p78Mk762YnZdsuzFV018NoG1Sj1SCQvpSqa7XUaTam5vAGasABV9qXASMKnFMwMw==", + "dev": true, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@jridgewell/sourcemap-codec": { + "version": "1.4.14", + "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.4.14.tgz", + "integrity": "sha512-XPSJHWmi394fuUuzDnGz1wiKqWfo1yXecHQMRf2l6hztTO+nPru658AyDngaBe7isIxEkRsPR3FZh+s7iVa4Uw==", + "dev": true + }, + "node_modules/@jridgewell/trace-mapping": { + "version": "0.3.17", + "resolved": "https://registry.npmjs.org/@jridgewell/trace-mapping/-/trace-mapping-0.3.17.tgz", + "integrity": "sha512-MCNzAp77qzKca9+W/+I0+sEpaUnZoeasnghNeVc41VZCEKaCH73Vq3BZZ/SzWIgrqE4H4ceI+p+b6C0mHf9T4g==", + "dev": true, + "dependencies": { + "@jridgewell/resolve-uri": "3.1.0", + "@jridgewell/sourcemap-codec": "1.4.14" + } + }, + "node_modules/@jsdoc/salty": { + "version": "0.2.5", + "resolved": "https://registry.npmjs.org/@jsdoc/salty/-/salty-0.2.5.tgz", + "integrity": "sha512-TfRP53RqunNe2HBobVBJ0VLhK1HbfvBYeTC1ahnN64PWvyYyGebmMiPkuwvD9fpw2ZbkoPb8Q7mwy0aR8Z9rvw==", + "dev": true, + "dependencies": { + "lodash": "^4.17.21" + }, + "engines": { + "node": ">=v12.0.0" + } + }, + "node_modules/@mapbox/node-pre-gyp": { + "version": "1.0.10", + "resolved": "https://registry.npmjs.org/@mapbox/node-pre-gyp/-/node-pre-gyp-1.0.10.tgz", + "integrity": "sha512-4ySo4CjzStuprMwk35H5pPbkymjv1SF3jGLj6rAHp/xT/RF7TL7bd9CTm1xDY49K2qF7jmR/g7k+SkLETP6opA==", + "optional": true, + "dependencies": { + "detect-libc": "^2.0.0", + "https-proxy-agent": "^5.0.0", + "make-dir": "^3.1.0", + "node-fetch": "^2.6.7", + "nopt": "^5.0.0", + "npmlog": "^5.0.1", + "rimraf": "^3.0.2", + "semver": "^7.3.5", + "tar": "^6.1.11" + }, + "bin": { + "node-pre-gyp": "bin/node-pre-gyp" + } + }, + "node_modules/@node-red/editor-api": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/@node-red/editor-api/-/editor-api-2.2.3.tgz", + "integrity": "sha512-lSqxKyf5FfODGCPQoJVr3m6oHxwIWhOQ6q1fIIxkL5JGQLJ3X/F5Du/hkgNUIr5W670S+WW+rYdx109ifiz1ng==", + "dependencies": { + "@node-red/editor-client": "2.2.3", + "@node-red/util": "2.2.3", + "bcryptjs": "2.4.3", + "body-parser": "1.19.1", + "clone": "2.1.2", + "cors": "2.8.5", + "express": "4.17.2", + "express-session": "1.17.2", + "memorystore": "1.6.7", + "mime": "3.0.0", + "multer": "1.4.4", + "mustache": "4.2.0", + "oauth2orize": "1.11.1", + "passport": "0.5.2", + "passport-http-bearer": "1.0.1", + "passport-oauth2-client-password": "0.1.2", + "ws": "7.5.6" + }, + "optionalDependencies": { + "bcrypt": "5.0.1" + } + }, + "node_modules/@node-red/editor-client": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/@node-red/editor-client/-/editor-client-2.2.3.tgz", + "integrity": "sha512-rkxx1VDE4IRiWj9u1f3yxdrt4JUP4mBtPLaLOZfPn7gPvXPJAmzA6MTzeetLWFAopxUCuH4+xsrfl5sOh1RFwQ==" + }, + "node_modules/@node-red/nodes": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/@node-red/nodes/-/nodes-2.2.3.tgz", + "integrity": "sha512-LRaoYQgWYGT3vDfHV+xHi33w7n9KDbNe9u77mNkLAna/PHq2IVViFWtBLm4uxhrYMYs7NNr7r+3NnVyhLGQlAQ==", + "dependencies": { + "acorn": "8.7.0", + "acorn-walk": "8.2.0", + "ajv": "8.10.0", + "body-parser": "1.19.1", + "cheerio": "1.0.0-rc.10", + "content-type": "1.0.4", + "cookie": "0.4.2", + "cookie-parser": "1.4.6", + "cors": "2.8.5", + "cronosjs": "1.7.1", + "denque": "2.0.1", + "form-data": "4.0.0", + "fs-extra": "10.0.0", + "fs.notify": "0.0.4", + "got": "11.8.3", + "hash-sum": "2.0.0", + "hpagent": "0.1.2", + "https-proxy-agent": "5.0.0", + "iconv-lite": "0.6.3", + "is-utf8": "0.2.1", + "js-yaml": "3.14.1", + "media-typer": "1.1.0", + "mqtt": "4.3.5", + "multer": "1.4.4", + "mustache": "4.2.0", + "on-headers": "1.0.2", + "raw-body": "2.4.3", + "tough-cookie": "4.0.0", + "uuid": "8.3.2", + "ws": "7.5.6", + "xml2js": "0.4.23" + } + }, + "node_modules/@node-red/registry": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/@node-red/registry/-/registry-2.2.3.tgz", + "integrity": "sha512-fR//QJhqDGfq91lg9onknb13xhTXMCSTQRdR1FbvhgR5g0uS4fl7ZcKTBhWkp7OzUZ+pRInuuc6JeBZTaeLKQg==", + "dependencies": { + "@node-red/util": "2.2.3", + "clone": "2.1.2", + "fs-extra": "10.0.0", + "semver": "7.3.5", + "tar": "6.1.11", + "uglify-js": "3.15.1" + } + }, + "node_modules/@node-red/runtime": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/@node-red/runtime/-/runtime-2.2.3.tgz", + "integrity": "sha512-NQrrhpg4daIdxkqET4n20az1sV5V5OvSpfkemKZbuXuicrPy4sGZ1OZ309v4QVbky97RgVrs7iv2Z1IRBXV8og==", + "dependencies": { + "@node-red/registry": "2.2.3", + "@node-red/util": "2.2.3", + "async-mutex": "0.3.2", + "clone": "2.1.2", + "express": "4.17.2", + "fs-extra": "10.0.0", + "json-stringify-safe": "5.0.1" + } + }, + "node_modules/@node-red/util": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/@node-red/util/-/util-2.2.3.tgz", + "integrity": "sha512-N/3US+wwa3mVm3jkSV/QPfVIRdQjaXyChgRPm9UAZbls6TrxJFfZxZefQt4xNKoDYQkHoFNN65Q29RtV0GfbbA==", + "dependencies": { + "fs-extra": "10.0.0", + "i18next": "21.6.11", + "json-stringify-safe": "5.0.1", + "jsonata": "1.8.6", + "lodash.clonedeep": "^4.5.0", + "moment-timezone": "0.5.34" + } + }, + "node_modules/@prantlf/jsonlint": { + "version": "10.2.0", + "resolved": "https://registry.npmjs.org/@prantlf/jsonlint/-/jsonlint-10.2.0.tgz", + "integrity": "sha512-KMFfds0peWLLfCu3bhClTiEN0tdj/Z86QJvn1awKHws6r+Sx6T3a44Eadz6OvqN6ZpsRkqaRpZxqddvvDAdDZQ==", + "dev": true, + "dependencies": { + "ajv": "6.10.2", + "commander": "4.0.1" + }, + "bin": { + "jsonlint": "lib/cli.js" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/@prantlf/jsonlint/node_modules/ajv": { + "version": "6.10.2", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-6.10.2.tgz", + "integrity": "sha512-TXtUUEYHuaTEbLZWIKUr5pmBuhDLy+8KYtPYdcV8qC+pOZL+NKqYwvWSRrVXHn+ZmRRAu8vJTAznH7Oag6RVRw==", + "dev": true, + "dependencies": { + "fast-deep-equal": "^2.0.1", + "fast-json-stable-stringify": "^2.0.0", + "json-schema-traverse": "^0.4.1", + "uri-js": "^4.2.2" + } + }, + "node_modules/@prantlf/jsonlint/node_modules/fast-deep-equal": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-2.0.1.tgz", + "integrity": "sha512-bCK/2Z4zLidyB4ReuIsvALH6w31YfAQDmXMqMx6FyfHqvBxtjC0eRumeSu4Bs3XtXwpyIywtSTrVT99BxY1f9w==", + "dev": true + }, + "node_modules/@prantlf/jsonlint/node_modules/json-schema-traverse": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-0.4.1.tgz", + "integrity": "sha512-xbbCH5dCYU5T8LcEhhuh7HJ88HXuW3qsI3Y0zOZFKfZEHcpWiHU/Jxzk629Brsab/mMiHQti9wMP+845RPe3Vg==", + "dev": true + }, + "node_modules/@sindresorhus/is": { + "version": "4.6.0", + "resolved": "https://registry.npmjs.org/@sindresorhus/is/-/is-4.6.0.tgz", + "integrity": "sha512-t09vSN3MdfsyCHoFcTRCH/iUtG7OJ0CsjzB8cjAmKc/va/kIgeDI/TxsigdncE/4be734m0cvIYwNaV4i2XqAw==", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sindresorhus/is?sponsor=1" + } + }, + "node_modules/@sinonjs/commons": { + "version": "1.8.6", + "resolved": "https://registry.npmjs.org/@sinonjs/commons/-/commons-1.8.6.tgz", + "integrity": "sha512-Ky+XkAkqPZSm3NLBeUng77EBQl3cmeJhITaGHdYH8kjVB+aun3S4XBRti2zt17mtt0mIUDiNxYeoJm6drVvBJQ==", + "dev": true, + "dependencies": { + "type-detect": "4.0.8" + } + }, + "node_modules/@sinonjs/fake-timers": { + "version": "7.1.2", + "resolved": "https://registry.npmjs.org/@sinonjs/fake-timers/-/fake-timers-7.1.2.tgz", + "integrity": "sha512-iQADsW4LBMISqZ6Ci1dupJL9pprqwcVFTcOsEmQOEhW+KLCVn/Y4Jrvg2k19fIHCp+iFprriYPTdRcQR8NbUPg==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^1.7.0" + } + }, + "node_modules/@sinonjs/samsam": { + "version": "6.1.3", + "resolved": "https://registry.npmjs.org/@sinonjs/samsam/-/samsam-6.1.3.tgz", + "integrity": "sha512-nhOb2dWPeb1sd3IQXL/dVPnKHDOAFfvichtBf4xV00/rU1QbPCQqKMbvIheIjqwVjh7qIgf2AHTHi391yMOMpQ==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^1.6.0", + "lodash.get": "^4.4.2", + "type-detect": "^4.0.8" + } + }, + "node_modules/@sinonjs/text-encoding": { + "version": "0.7.2", + "resolved": "https://registry.npmjs.org/@sinonjs/text-encoding/-/text-encoding-0.7.2.tgz", + "integrity": "sha512-sXXKG+uL9IrKqViTtao2Ws6dy0znu9sOaP1di/jKGW1M6VssO8vlpXCQcpZ+jisQ1tTFAC5Jo/EOzFbggBagFQ==", + "dev": true + }, + "node_modules/@szmarczak/http-timer": { + "version": "4.0.6", + "resolved": "https://registry.npmjs.org/@szmarczak/http-timer/-/http-timer-4.0.6.tgz", + "integrity": "sha512-4BAffykYOgO+5nzBWYwE3W90sBgLJoUPRWWcL8wlyiM8IB8ipJz3UMJ9KXQd1RKQXpKp8Tutn80HZtWsu2u76w==", + "dependencies": { + "defer-to-connect": "^2.0.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/@types/cacheable-request": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/@types/cacheable-request/-/cacheable-request-6.0.3.tgz", + "integrity": "sha512-IQ3EbTzGxIigb1I3qPZc1rWJnH0BmSKv5QYTalEwweFvyBDLSAe24zP0le/hyi7ecGfZVlIVAg4BZqb8WBwKqw==", + "dependencies": { + "@types/http-cache-semantics": "*", + "@types/keyv": "^3.1.4", + "@types/node": "*", + "@types/responselike": "^1.0.0" + } + }, + "node_modules/@types/http-cache-semantics": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/@types/http-cache-semantics/-/http-cache-semantics-4.0.1.tgz", + "integrity": "sha512-SZs7ekbP8CN0txVG2xVRH6EgKmEm31BOxA07vkFaETzZz1xh+cbt8BcI0slpymvwhx5dlFnQG2rTlPVQn+iRPQ==" + }, + "node_modules/@types/keyv": { + "version": "3.1.4", + "resolved": "https://registry.npmjs.org/@types/keyv/-/keyv-3.1.4.tgz", + "integrity": "sha512-BQ5aZNSCpj7D6K2ksrRCTmKRLEpnPvWDiLPfoGyhZ++8YtiK9d/3DBKPJgry359X/P1PfruyYwvnvwFjuEiEIg==", + "dependencies": { + "@types/node": "*" + } + }, + "node_modules/@types/linkify-it": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/linkify-it/-/linkify-it-3.0.2.tgz", + "integrity": "sha512-HZQYqbiFVWufzCwexrvh694SOim8z2d+xJl5UNamcvQFejLY/2YUtzXHYi3cHdI7PMlS8ejH2slRAOJQ32aNbA==", + "dev": true + }, + "node_modules/@types/markdown-it": { + "version": "12.2.3", + "resolved": "https://registry.npmjs.org/@types/markdown-it/-/markdown-it-12.2.3.tgz", + "integrity": "sha512-GKMHFfv3458yYy+v/N8gjufHO6MSZKCOXpZc5GXIWWy8uldwfmPn98vp81gZ5f9SVw8YYBctgfJ22a2d7AOMeQ==", + "dev": true, + "dependencies": { + "@types/linkify-it": "*", + "@types/mdurl": "*" + } + }, + "node_modules/@types/mdurl": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/@types/mdurl/-/mdurl-1.0.2.tgz", + "integrity": "sha512-eC4U9MlIcu2q0KQmXszyn5Akca/0jrQmwDRgpAMJai7qBWq4amIQhZyNau4VYGtCeALvW1/NtjzJJ567aZxfKA==", + "dev": true + }, + "node_modules/@types/node": { + "version": "18.15.5", + "resolved": "https://registry.npmjs.org/@types/node/-/node-18.15.5.tgz", + "integrity": "sha512-Ark2WDjjZO7GmvsyFFf81MXuGTA/d6oP38anyxWOL6EREyBKAxKoFHwBhaZxCfLRLpO8JgVXwqOwSwa7jRcjew==" + }, + "node_modules/@types/normalize-package-data": { + "version": "2.4.1", + "resolved": "https://registry.npmjs.org/@types/normalize-package-data/-/normalize-package-data-2.4.1.tgz", + "integrity": "sha512-Gj7cI7z+98M282Tqmp2K5EIsoouUEzbBJhQQzDE3jSIRk6r9gsz0oUokqIUR4u1R3dMHo0pDHM7sNOHyhulypw==", + "dev": true + }, + "node_modules/@types/responselike": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/@types/responselike/-/responselike-1.0.0.tgz", + "integrity": "sha512-85Y2BjiufFzaMIlvJDvTTB8Fxl2xfLo4HgmHzVBz08w4wDePCTjYw66PdrolO0kzli3yam/YCgRufyo1DdQVTA==", + "dependencies": { + "@types/node": "*" + } + }, + "node_modules/@ungap/promise-all-settled": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@ungap/promise-all-settled/-/promise-all-settled-1.1.2.tgz", + "integrity": "sha512-sL/cEvJWAnClXw0wHk85/2L0G6Sj8UB0Ctc1TEMbKSsmpRosqhwj9gWgFRZSrBr2f9tiXISwNhCPmlfqUqyb9Q==", + "dev": true + }, + "node_modules/abbrev": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/abbrev/-/abbrev-1.1.1.tgz", + "integrity": "sha512-nne9/IiQ/hzIhY6pdDnbBtz7DjPTKrY00P/zvPSm5pOFkl6xuGrGnXn/VtTNNfNtAfZ9/1RtehkszU9qcTii0Q==" + }, + "node_modules/accepts": { + "version": "1.3.8", + "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz", + "integrity": "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw==", + "dependencies": { + "mime-types": "~2.1.34", + "negotiator": "0.6.3" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/acorn": { + "version": "8.7.0", + "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.7.0.tgz", + "integrity": "sha512-V/LGr1APy+PXIwKebEWrkZPwoeoF+w1jiOBUmuxuiUIaOHtob8Qc9BTrYo7VuI5fR8tqsy+buA2WFooR5olqvQ==", + "bin": { + "acorn": "bin/acorn" + }, + "engines": { + "node": ">=0.4.0" + } + }, + "node_modules/acorn-walk": { + "version": "8.2.0", + "resolved": "https://registry.npmjs.org/acorn-walk/-/acorn-walk-8.2.0.tgz", + "integrity": "sha512-k+iyHEuPgSw6SbuDpGQM+06HQUa04DZ3o+F6CSzXMvvI5KMvnaEqXe+YVe555R9nn6GPt404fos4wcgpw12SDA==", + "engines": { + "node": ">=0.4.0" + } + }, + "node_modules/adm-zip": { + "version": "0.5.10", + "resolved": "https://registry.npmjs.org/adm-zip/-/adm-zip-0.5.10.tgz", + "integrity": "sha512-x0HvcHqVJNTPk/Bw8JbLWlWoo6Wwnsug0fnYYro1HBrjxZ3G7/AZk7Ahv8JwDe1uIcz8eBqvu86FuF1POiG7vQ==", + "dev": true, + "engines": { + "node": ">=6.0" + } + }, + "node_modules/agent-base": { + "version": "6.0.2", + "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz", + "integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==", + "dependencies": { + "debug": "4" + }, + "engines": { + "node": ">= 6.0.0" + } + }, + "node_modules/agent-base/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/agent-base/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==" + }, + "node_modules/ajv": { + "version": "8.10.0", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.10.0.tgz", + "integrity": "sha512-bzqAEZOjkrUMl2afH8dknrq5KEk2SrwdBROR+vH1EKVQTqaUbJVPdc/gEdggTMM0Se+s+Ja4ju4TlNcStKl2Hw==", + "dependencies": { + "fast-deep-equal": "^3.1.1", + "json-schema-traverse": "^1.0.0", + "require-from-string": "^2.0.2", + "uri-js": "^4.2.2" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/epoberezkin" + } + }, + "node_modules/ansi-align": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/ansi-align/-/ansi-align-3.0.1.tgz", + "integrity": "sha512-IOfwwBF5iczOjp/WeY4YxyjqAFMQoZufdQWDd19SEExbVLNXqvpzSJ/M7Za4/sCPmQ0+GRquoA7bGcINcxew6w==", + "dev": true, + "dependencies": { + "string-width": "^4.1.0" + } + }, + "node_modules/ansi-colors": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/ansi-colors/-/ansi-colors-4.1.1.tgz", + "integrity": "sha512-JoX0apGbHaUJBNl6yF+p6JAFYZ666/hhCGKN5t9QFjbJQKUU/g8MNbFDbvfrgKXvI1QpZplPOnwIo99lX/AAmA==", + "engines": { + "node": ">=6" + } + }, + "node_modules/ansi-escape-sequences": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/ansi-escape-sequences/-/ansi-escape-sequences-4.1.0.tgz", + "integrity": "sha512-dzW9kHxH011uBsidTXd14JXgzye/YLb2LzeKZ4bsgl/Knwx8AtbSFkkGxagdNOoh0DlqHCmfiEjWKBaqjOanVw==", + "dev": true, + "dependencies": { + "array-back": "^3.0.1" + }, + "engines": { + "node": ">=8.0.0" + } + }, + "node_modules/ansi-escape-sequences/node_modules/array-back": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-3.1.0.tgz", + "integrity": "sha512-TkuxA4UCOvxuDK6NZYXCalszEzj+TLszyASooky+i742l9TqsOdYCMJJupxRic61hwquNtppB3hgcuq9SVSH1Q==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/ansi-regex": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz", + "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==", + "devOptional": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/ansi-styles": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz", + "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==", + "dev": true, + "dependencies": { + "color-convert": "^2.0.1" + }, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/chalk/ansi-styles?sponsor=1" + } + }, + "node_modules/anymatch": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/anymatch/-/anymatch-3.1.3.tgz", + "integrity": "sha512-KMReFUr0B4t+D+OBkjR3KYqvocp2XaSzO55UcB6mgQMd3KbcE+mWTyvVV7D/zsdEbNnV6acZUutkiHQXvTr1Rw==", + "dev": true, + "dependencies": { + "normalize-path": "^3.0.0", + "picomatch": "^2.0.4" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/append-field": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/append-field/-/append-field-1.0.0.tgz", + "integrity": "sha512-klpgFSWLW1ZEs8svjfb7g4qWY0YS5imI82dTg+QahUvJ8YqAY0P10Uk8tTyh9ZGuYEZEMaeJYCF5BFuX552hsw==" + }, + "node_modules/append-transform": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/append-transform/-/append-transform-1.0.0.tgz", + "integrity": "sha512-P009oYkeHyU742iSZJzZZywj4QRJdnTWffaKuJQLablCZ1uz6/cW4yaRgcDaoQ+uwOxxnt0gRUcwfsNP2ri0gw==", + "dev": true, + "dependencies": { + "default-require-extensions": "^2.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/aproba": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/aproba/-/aproba-2.0.0.tgz", + "integrity": "sha512-lYe4Gx7QT+MKGbDsA+Z+he/Wtef0BiwDOlK/XkBrdfsh9J/jPPXbX0tE9x9cl27Tmu5gg3QUbUrQYa/y+KOHPQ==", + "optional": true + }, + "node_modules/archiver": { + "version": "5.3.1", + "resolved": "https://registry.npmjs.org/archiver/-/archiver-5.3.1.tgz", + "integrity": "sha512-8KyabkmbYrH+9ibcTScQ1xCJC/CGcugdVIwB+53f5sZziXgwUh3iXlAlANMxcZyDEfTHMe6+Z5FofV8nopXP7w==", + "dev": true, + "dependencies": { + "archiver-utils": "^2.1.0", + "async": "^3.2.3", + "buffer-crc32": "^0.2.1", + "readable-stream": "^3.6.0", + "readdir-glob": "^1.0.0", + "tar-stream": "^2.2.0", + "zip-stream": "^4.1.0" + }, + "engines": { + "node": ">= 10" + } + }, + "node_modules/archiver-utils": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/archiver-utils/-/archiver-utils-2.1.0.tgz", + "integrity": "sha512-bEL/yUb/fNNiNTuUz979Z0Yg5L+LzLxGJz8x79lYmR54fmTIb6ob/hNQgkQnIUDWIFjZVQwl9Xs356I6BAMHfw==", + "dev": true, + "dependencies": { + "glob": "^7.1.4", + "graceful-fs": "^4.2.0", + "lazystream": "^1.0.0", + "lodash.defaults": "^4.2.0", + "lodash.difference": "^4.5.0", + "lodash.flatten": "^4.4.0", + "lodash.isplainobject": "^4.0.6", + "lodash.union": "^4.6.0", + "normalize-path": "^3.0.0", + "readable-stream": "^2.0.0" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/archiver-utils/node_modules/readable-stream": { + "version": "2.3.8", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.8.tgz", + "integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==", + "dev": true, + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.3", + "isarray": "~1.0.0", + "process-nextick-args": "~2.0.0", + "safe-buffer": "~5.1.1", + "string_decoder": "~1.1.1", + "util-deprecate": "~1.0.1" + } + }, + "node_modules/archiver-utils/node_modules/string_decoder": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz", + "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==", + "dev": true, + "dependencies": { + "safe-buffer": "~5.1.0" + } + }, + "node_modules/archiver/node_modules/async": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/async/-/async-3.2.4.tgz", + "integrity": "sha512-iAB+JbDEGXhyIUavoDl9WP/Jj106Kz9DEn1DPgYw5ruDn0e3Wgi3sKFm55sASdGBNOQB8F59d9qQ7deqrHA8wQ==", + "dev": true + }, + "node_modules/archy": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/archy/-/archy-1.0.0.tgz", + "integrity": "sha512-Xg+9RwCg/0p32teKdGMPTPnVXKD0w3DfHnFTficozsAgsvq2XenPJq/MYpzzQ/v8zrOyJn6Ds39VA4JIDwFfqw==", + "dev": true + }, + "node_modules/are-we-there-yet": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/are-we-there-yet/-/are-we-there-yet-2.0.0.tgz", + "integrity": "sha512-Ci/qENmwHnsYo9xKIcUJN5LeDKdJ6R1Z1j9V/J5wyq8nh/mYPEpIKJbBZXtZjG04HiK7zV/p6Vs9952MrMeUIw==", + "optional": true, + "dependencies": { + "delegates": "^1.0.0", + "readable-stream": "^3.6.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/argparse": { + "version": "1.0.10", + "resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz", + "integrity": "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg==", + "dependencies": { + "sprintf-js": "~1.0.2" + } + }, + "node_modules/args": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/args/-/args-5.0.1.tgz", + "integrity": "sha512-1kqmFCFsPffavQFGt8OxJdIcETti99kySRUPMpOhaGjL6mRJn8HFU1OxKY5bMqfZKUwTQc1mZkAjmGYaVOHFtQ==", + "dev": true, + "dependencies": { + "camelcase": "5.0.0", + "chalk": "2.4.2", + "leven": "2.1.0", + "mri": "1.1.4" + }, + "engines": { + "node": ">= 6.0.0" + } + }, + "node_modules/args/node_modules/ansi-styles": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz", + "integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==", + "dev": true, + "dependencies": { + "color-convert": "^1.9.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/args/node_modules/chalk": { + "version": "2.4.2", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz", + "integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==", + "dev": true, + "dependencies": { + "ansi-styles": "^3.2.1", + "escape-string-regexp": "^1.0.5", + "supports-color": "^5.3.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/args/node_modules/color-convert": { + "version": "1.9.3", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.3.tgz", + "integrity": "sha512-QfAUtd+vFdAtFQcC8CCyYt1fYWxSqAiK2cSD6zDB8N3cpsEBAvRxp9zOGg6G/SHHJYAT88/az/IuDGALsNVbGg==", + "dev": true, + "dependencies": { + "color-name": "1.1.3" + } + }, + "node_modules/args/node_modules/color-name": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz", + "integrity": "sha512-72fSenhMw2HZMTVHeCA9KCmpEIbzWiQsjN+BHcBbS9vr1mtt+vJjPdksIBNUmKAW8TFUDPJK5SUU3QhE9NEXDw==", + "dev": true + }, + "node_modules/args/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/args/node_modules/has-flag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", + "integrity": "sha512-sKJf1+ceQBr4SMkvQnBDNDtf4TXpVhVGateu0t918bl30FnbE2m4vNLX+VWe/dpjlb+HugGYzW7uQXH98HPEYw==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/args/node_modules/supports-color": { + "version": "5.5.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz", + "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==", + "dev": true, + "dependencies": { + "has-flag": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/array-back": { + "version": "6.2.2", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-6.2.2.tgz", + "integrity": "sha512-gUAZ7HPyb4SJczXAMUXMGAvI976JoK3qEx9v1FTmeYuJj0IBiaKttG1ydtGKdkfqWkIkouke7nG8ufGy77+Cvw==", + "dev": true, + "engines": { + "node": ">=12.17" + } + }, + "node_modules/array-each": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/array-each/-/array-each-1.0.1.tgz", + "integrity": "sha512-zHjL5SZa68hkKHBFBK6DJCTtr9sfTCPCaph/L7tMSLcTFgy+zX7E+6q5UArbtOtMBCtxdICpfTCspRse+ywyXA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/array-flatten": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz", + "integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==" + }, + "node_modules/array-slice": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/array-slice/-/array-slice-1.1.0.tgz", + "integrity": "sha512-B1qMD3RBP7O8o0H2KbrXDyB0IccejMF15+87Lvlor12ONPRHP6gTjXMNkt/d3ZuOGbAe66hFmaCfECI24Ufp6w==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/arrify": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/arrify/-/arrify-2.0.1.tgz", + "integrity": "sha512-3duEwti880xqi4eAMN8AyR4a0ByT90zoYdLlevfrvU43vb0YZwZVfxOgxWrLXXXpyugL0hNZc9G6BiB5B3nUug==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/asap": { + "version": "2.0.6", + "resolved": "https://registry.npmjs.org/asap/-/asap-2.0.6.tgz", + "integrity": "sha512-BSHWgDSAiKs50o2Re8ppvp3seVHXSRM44cdSsT9FfNEUUZLOGWVCsiWaRPWM1Znn+mqZ1OfVZ3z3DWEzSp7hRA==", + "dev": true + }, + "node_modules/async": { + "version": "0.1.22", + "resolved": "https://registry.npmjs.org/async/-/async-0.1.22.tgz", + "integrity": "sha512-2tEzliJmf5fHNafNwQLJXUasGzQCVctvsNkXmnlELHwypU0p08/rHohYvkqKIjyXpx+0rkrYv6QbhJ+UF4QkBg==", + "engines": { + "node": "*" + } + }, + "node_modules/async-mutex": { + "version": "0.3.2", + "resolved": "https://registry.npmjs.org/async-mutex/-/async-mutex-0.3.2.tgz", + "integrity": "sha512-HuTK7E7MT7jZEh1P9GtRW9+aTWiDWWi9InbZ5hjxrnRa39KS4BW04+xLBhYNS2aXhHUIKZSw3gj4Pn1pj+qGAA==", + "dependencies": { + "tslib": "^2.3.1" + } + }, + "node_modules/asynckit": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", + "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==" + }, + "node_modules/axios": { + "version": "0.27.0", + "resolved": "https://registry.npmjs.org/axios/-/axios-0.27.0.tgz", + "integrity": "sha512-XV/WrPxXfzgZ8j4lcB5i6LyaXmi90yetmV/Fem0kmglGx+mpY06CiweL3YxU6wOTNLmqLUePW4G8h45nGZ/+pA==", + "deprecated": "Formdata complete broken, incorrect build size", + "dependencies": { + "follow-redirects": "^1.14.9", + "form-data": "^4.0.0" + } + }, + "node_modules/balanced-match": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz", + "integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==" + }, + "node_modules/base64-js": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", + "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ] + }, + "node_modules/basic-auth": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/basic-auth/-/basic-auth-2.0.1.tgz", + "integrity": "sha512-NF+epuEdnUYVlGuhaxbbq+dvJttwLnGY+YixlXlME5KpQ5W3CnXA5cVTneY3SPbPDRkcjMbifrwmFYcClgOZeg==", + "dependencies": { + "safe-buffer": "5.1.2" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/basic-auth-parser": { + "version": "0.0.2", + "resolved": "https://registry.npmjs.org/basic-auth-parser/-/basic-auth-parser-0.0.2.tgz", + "integrity": "sha512-Y7OBvWn+JnW45JWHLY6ybYub2k9cXCMrtCyO1Hds2s6eqClqWhPnOQpgXUPjAiMHj+A8TEPIQQ1dYENnJoBOHQ==", + "dev": true + }, + "node_modules/bcrypt": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/bcrypt/-/bcrypt-5.0.1.tgz", + "integrity": "sha512-9BTgmrhZM2t1bNuDtrtIMVSmmxZBrJ71n8Wg+YgdjHuIWYF7SjjmCPZFB+/5i/o/PIeRpwVJR3P+NrpIItUjqw==", + "hasInstallScript": true, + "optional": true, + "dependencies": { + "@mapbox/node-pre-gyp": "^1.0.0", + "node-addon-api": "^3.1.0" + }, + "engines": { + "node": ">= 10.0.0" + } + }, + "node_modules/bcryptjs": { + "version": "2.4.3", + "resolved": "https://registry.npmjs.org/bcryptjs/-/bcryptjs-2.4.3.tgz", + "integrity": "sha512-V/Hy/X9Vt7f3BbPJEi8BdVFMByHi+jNXrYkW3huaybV/kQ0KJg0Y6PkEMbn+zeT+i+SiKZ/HMqJGIIt4LZDqNQ==" + }, + "node_modules/binary-extensions": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.2.0.tgz", + "integrity": "sha512-jDctJ/IVQbZoJykoeHbhXpOlNBqGNcwXJKJog42E5HDPUwQTSdjCHdihjj0DlnheQ7blbT6dHOafNAiS8ooQKA==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/bl": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz", + "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==", + "dependencies": { + "buffer": "^5.5.0", + "inherits": "^2.0.4", + "readable-stream": "^3.4.0" + } + }, + "node_modules/bluebird": { + "version": "3.7.2", + "resolved": "https://registry.npmjs.org/bluebird/-/bluebird-3.7.2.tgz", + "integrity": "sha512-XpNj6GDQzdfW+r2Wnn7xiSAd7TM3jzkxGXBGTtWKuSXv1xUV+azxAm8jdWZN06QTQk+2N2XB9jRDkvbmQmcRtg==", + "dev": true + }, + "node_modules/body": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/body/-/body-5.1.0.tgz", + "integrity": "sha512-chUsBxGRtuElD6fmw1gHLpvnKdVLK302peeFa9ZqAEk8TyzZ3fygLyUEDDPTJvL9+Bor0dIwn6ePOsRM2y0zQQ==", + "dev": true, + "dependencies": { + "continuable-cache": "^0.3.1", + "error": "^7.0.0", + "raw-body": "~1.1.0", + "safe-json-parse": "~1.0.1" + } + }, + "node_modules/body-parser": { + "version": "1.19.1", + "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.19.1.tgz", + "integrity": "sha512-8ljfQi5eBk8EJfECMrgqNGWPEY5jWP+1IzkzkGdFFEwFQZZyaZ21UqdaHktgiMlH0xLHqIFtE/u2OYE5dOtViA==", + "dependencies": { + "bytes": "3.1.1", + "content-type": "~1.0.4", + "debug": "2.6.9", + "depd": "~1.1.2", + "http-errors": "1.8.1", + "iconv-lite": "0.4.24", + "on-finished": "~2.3.0", + "qs": "6.9.6", + "raw-body": "2.4.2", + "type-is": "~1.6.18" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/body-parser/node_modules/iconv-lite": { + "version": "0.4.24", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz", + "integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==", + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/body-parser/node_modules/raw-body": { + "version": "2.4.2", + "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.4.2.tgz", + "integrity": "sha512-RPMAFUJP19WIet/99ngh6Iv8fzAbqum4Li7AD6DtGaW2RpMB/11xDoalPiJMTbu6I3hkbMVkATvZrqb9EEqeeQ==", + "dependencies": { + "bytes": "3.1.1", + "http-errors": "1.8.1", + "iconv-lite": "0.4.24", + "unpipe": "1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/body/node_modules/bytes": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/bytes/-/bytes-1.0.0.tgz", + "integrity": "sha512-/x68VkHLeTl3/Ll8IvxdwzhrT+IyKc52e/oyHhA2RwqPqswSnjVbSddfPRwAsJtbilMAPSRWwAlpxdYsSWOTKQ==", + "dev": true + }, + "node_modules/body/node_modules/raw-body": { + "version": "1.1.7", + "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-1.1.7.tgz", + "integrity": "sha512-WmJJU2e9Y6M5UzTOkHaM7xJGAPQD8PNzx3bAd2+uhZAim6wDk6dAZxPVYLF67XhbR4hmKGh33Lpmh4XWrCH5Mg==", + "dev": true, + "dependencies": { + "bytes": "1", + "string_decoder": "0.10" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/body/node_modules/string_decoder": { + "version": "0.10.31", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz", + "integrity": "sha512-ev2QzSzWPYmy9GuqfIVildA4OdcGLeFZQrq5ys6RtiuF+RQQiZWr8TZNyAcuVXyQRYfEO+MsoB/1BuQVhOJuoQ==", + "dev": true + }, + "node_modules/boolbase": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz", + "integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==" + }, + "node_modules/boxen": { + "version": "5.1.2", + "resolved": "https://registry.npmjs.org/boxen/-/boxen-5.1.2.tgz", + "integrity": "sha512-9gYgQKXx+1nP8mP7CzFyaUARhg7D3n1dF/FnErWmu9l6JvGpNUN278h0aSb+QjoiKSWG+iZ3uHrcqk0qrY9RQQ==", + "dev": true, + "dependencies": { + "ansi-align": "^3.0.0", + "camelcase": "^6.2.0", + "chalk": "^4.1.0", + "cli-boxes": "^2.2.1", + "string-width": "^4.2.2", + "type-fest": "^0.20.2", + "widest-line": "^3.1.0", + "wrap-ansi": "^7.0.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/boxen/node_modules/camelcase": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/camelcase/-/camelcase-6.3.0.tgz", + "integrity": "sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/boxen/node_modules/type-fest": { + "version": "0.20.2", + "resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.20.2.tgz", + "integrity": "sha512-Ne+eE4r0/iWnpAxD852z3A+N0Bt5RN//NjJwRd2VFHEmrywxf5vsZlh4R6lixl6B+wz/8d+maTSAkN1FIkI3LQ==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/brace-expansion": { + "version": "1.1.11", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz", + "integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==", + "dependencies": { + "balanced-match": "^1.0.0", + "concat-map": "0.0.1" + } + }, + "node_modules/braces": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/braces/-/braces-3.0.2.tgz", + "integrity": "sha512-b8um+L1RzM3WDSzvhm6gIz1yfTbBt6YTlcEKAvsmqCZZFw46z626lVj9j1yEPW33H5H+lBQpZMP1k8l+78Ha0A==", + "dev": true, + "dependencies": { + "fill-range": "^7.0.1" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/browser-stdout": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/browser-stdout/-/browser-stdout-1.3.1.tgz", + "integrity": "sha512-qhAVI1+Av2X7qelOfAIYwXONood6XlZE/fXaBSmW/T5SzLAmCgzi+eiWE7fUvbHaeNBQH13UftjpXxsfLkMpgw==", + "dev": true + }, + "node_modules/buffer": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz", + "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "dependencies": { + "base64-js": "^1.3.1", + "ieee754": "^1.1.13" + } + }, + "node_modules/buffer-crc32": { + "version": "0.2.13", + "resolved": "https://registry.npmjs.org/buffer-crc32/-/buffer-crc32-0.2.13.tgz", + "integrity": "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ==", + "dev": true, + "engines": { + "node": "*" + } + }, + "node_modules/buffer-from": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz", + "integrity": "sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==" + }, + "node_modules/busboy": { + "version": "0.2.14", + "resolved": "https://registry.npmjs.org/busboy/-/busboy-0.2.14.tgz", + "integrity": "sha512-InWFDomvlkEj+xWLBfU3AvnbVYqeTWmQopiW0tWWEy5yehYm2YkGEc59sUmw/4ty5Zj/b0WHGs1LgecuBSBGrg==", + "dependencies": { + "dicer": "0.2.5", + "readable-stream": "1.1.x" + }, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/busboy/node_modules/isarray": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", + "integrity": "sha512-D2S+3GLxWH+uhrNEcoh/fnmYeP8E8/zHl644d/jdA0g2uyXvy3sb0qxotE+ne0LtccHknQzWwZEzhak7oJ0COQ==" + }, + "node_modules/busboy/node_modules/readable-stream": { + "version": "1.1.14", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz", + "integrity": "sha512-+MeVjFf4L44XUkhM1eYbD8fyEsxcV81pqMSR5gblfcLCHfZvbrqy4/qYHE+/R5HoBUT11WV5O08Cr1n3YXkWVQ==", + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.1", + "isarray": "0.0.1", + "string_decoder": "~0.10.x" + } + }, + "node_modules/busboy/node_modules/string_decoder": { + "version": "0.10.31", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz", + "integrity": "sha512-ev2QzSzWPYmy9GuqfIVildA4OdcGLeFZQrq5ys6RtiuF+RQQiZWr8TZNyAcuVXyQRYfEO+MsoB/1BuQVhOJuoQ==" + }, + "node_modules/bytes": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.1.tgz", + "integrity": "sha512-dWe4nWO/ruEOY7HkUJ5gFt1DCFV9zPRoJr8pV0/ASQermOZjtq8jMjOprC0Kd10GLN+l7xaUPvxzJFWtxGu8Fg==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/cache-point": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/cache-point/-/cache-point-2.0.0.tgz", + "integrity": "sha512-4gkeHlFpSKgm3vm2gJN5sPqfmijYRFYCQ6tv5cLw0xVmT6r1z1vd4FNnpuOREco3cBs1G709sZ72LdgddKvL5w==", + "dev": true, + "dependencies": { + "array-back": "^4.0.1", + "fs-then-native": "^2.0.0", + "mkdirp2": "^1.0.4" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/cache-point/node_modules/array-back": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-4.0.2.tgz", + "integrity": "sha512-NbdMezxqf94cnNfWLL7V/im0Ub+Anbb0IoZhvzie8+4HJ4nMQuzHuy49FkGYCJK2yAloZ3meiB6AVMClbrI1vg==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/cacheable-lookup": { + "version": "5.0.4", + "resolved": "https://registry.npmjs.org/cacheable-lookup/-/cacheable-lookup-5.0.4.tgz", + "integrity": "sha512-2/kNscPhpcxrOigMZzbiWF7dz8ilhb/nIHU3EyZiXWXpeq/au8qJ8VhdftMkty3n7Gj6HIGalQG8oiBNB3AJgA==", + "engines": { + "node": ">=10.6.0" + } + }, + "node_modules/cacheable-request": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/cacheable-request/-/cacheable-request-7.0.2.tgz", + "integrity": "sha512-pouW8/FmiPQbuGpkXQ9BAPv/Mo5xDGANgSNXzTzJ8DrKGuXOssM4wIQRjfanNRh3Yu5cfYPvcorqbhg2KIJtew==", + "dependencies": { + "clone-response": "^1.0.2", + "get-stream": "^5.1.0", + "http-cache-semantics": "^4.0.0", + "keyv": "^4.0.0", + "lowercase-keys": "^2.0.0", + "normalize-url": "^6.0.1", + "responselike": "^2.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/caching-transform": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/caching-transform/-/caching-transform-3.0.2.tgz", + "integrity": "sha512-Mtgcv3lh3U0zRii/6qVgQODdPA4G3zhG+jtbCWj39RXuUFTMzH0vcdMtaJS1jPowd+It2Pqr6y3NJMQqOqCE2w==", + "dev": true, + "dependencies": { + "hasha": "^3.0.0", + "make-dir": "^2.0.0", + "package-hash": "^3.0.0", + "write-file-atomic": "^2.4.2" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/caching-transform/node_modules/make-dir": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/make-dir/-/make-dir-2.1.0.tgz", + "integrity": "sha512-LS9X+dc8KLxXCb8dni79fLIIUA5VyZoyjSMCwTluaXA0o27cCK0bhXkpgw+sTXVpPy/lSO57ilRixqk0vDmtRA==", + "dev": true, + "dependencies": { + "pify": "^4.0.1", + "semver": "^5.6.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/caching-transform/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/call-bind": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/call-bind/-/call-bind-1.0.2.tgz", + "integrity": "sha512-7O+FbCihrB5WGbFYesctwmTKae6rOiIzmz1icreWJ+0aA7LJfuqhEso2T9ncpcFtzMQtzXf2QGGueWJGTYsqrA==", + "dev": true, + "dependencies": { + "function-bind": "^1.1.1", + "get-intrinsic": "^1.0.2" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/camelcase": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/camelcase/-/camelcase-5.0.0.tgz", + "integrity": "sha512-faqwZqnWxbxn+F1d399ygeamQNy3lPp/H9H6rNrqYh4FSVCtcY+3cub1MxA8o9mDd55mM8Aghuu/kuyYA6VTsA==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/catharsis": { + "version": "0.9.0", + "resolved": "https://registry.npmjs.org/catharsis/-/catharsis-0.9.0.tgz", + "integrity": "sha512-prMTQVpcns/tzFgFVkVp6ak6RykZyWb3gu8ckUpd6YkTlacOd3DXGJjIpD4Q6zJirizvaiAjSSHlOsA+6sNh2A==", + "dev": true, + "dependencies": { + "lodash": "^4.17.15" + }, + "engines": { + "node": ">= 10" + } + }, + "node_modules/chalk": { + "version": "4.1.2", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz", + "integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==", + "dev": true, + "dependencies": { + "ansi-styles": "^4.1.0", + "supports-color": "^7.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/chalk/chalk?sponsor=1" + } + }, + "node_modules/cheerio": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/cheerio/-/cheerio-1.0.0-rc.10.tgz", + "integrity": "sha512-g0J0q/O6mW8z5zxQ3A8E8J1hUgp4SMOvEoW/x84OwyHKe/Zccz83PVT4y5Crcr530FV6NgmKI1qvGTKVl9XXVw==", + "dependencies": { + "cheerio-select": "^1.5.0", + "dom-serializer": "^1.3.2", + "domhandler": "^4.2.0", + "htmlparser2": "^6.1.0", + "parse5": "^6.0.1", + "parse5-htmlparser2-tree-adapter": "^6.0.1", + "tslib": "^2.2.0" + }, + "engines": { + "node": ">= 6" + }, + "funding": { + "url": "https://github.com/cheeriojs/cheerio?sponsor=1" + } + }, + "node_modules/cheerio-select": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/cheerio-select/-/cheerio-select-1.6.0.tgz", + "integrity": "sha512-eq0GdBvxVFbqWgmCm7M3XGs1I8oLy/nExUnh6oLqmBditPO9AqQJrkslDpMun/hZ0yyTs8L0m85OHp4ho6Qm9g==", + "dependencies": { + "css-select": "^4.3.0", + "css-what": "^6.0.1", + "domelementtype": "^2.2.0", + "domhandler": "^4.3.1", + "domutils": "^2.8.0" + }, + "funding": { + "url": "https://github.com/sponsors/fb55" + } + }, + "node_modules/chokidar": { + "version": "3.5.3", + "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-3.5.3.tgz", + "integrity": "sha512-Dr3sfKRP6oTcjf2JmUmFJfeVMvXBdegxB0iVQ5eb2V10uFJUCAS8OByZdVAyVb8xXNz3GjjTgj9kLWsZTqE6kw==", + "dev": true, + "funding": [ + { + "type": "individual", + "url": "https://paulmillr.com/funding/" + } + ], + "dependencies": { + "anymatch": "~3.1.2", + "braces": "~3.0.2", + "glob-parent": "~5.1.2", + "is-binary-path": "~2.1.0", + "is-glob": "~4.0.1", + "normalize-path": "~3.0.0", + "readdirp": "~3.6.0" + }, + "engines": { + "node": ">= 8.10.0" + }, + "optionalDependencies": { + "fsevents": "~2.3.2" + } + }, + "node_modules/chownr": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/chownr/-/chownr-2.0.0.tgz", + "integrity": "sha512-bIomtDF5KGpdogkLd9VspvFzk9KfpyyGlS8YFVZl7TGPBHL5snIOnxeshwVgPteQ9b4Eydl+pVbIyE1DcvCWgQ==", + "engines": { + "node": ">=10" + } + }, + "node_modules/ci-info": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/ci-info/-/ci-info-2.0.0.tgz", + "integrity": "sha512-5tK7EtrZ0N+OLFMthtqOj4fI2Jeb88C4CAZPu25LDVUgXJ0A3Js4PMGqrn0JU1W0Mh1/Z8wZzYPxqUrXeBboCQ==", + "dev": true + }, + "node_modules/cli": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/cli/-/cli-1.0.1.tgz", + "integrity": "sha512-41U72MB56TfUMGndAKK8vJ78eooOD4Z5NOL4xEfjc0c23s+6EYKXlXsmACBVclLP1yOfWCgEganVzddVrSNoTg==", + "dev": true, + "dependencies": { + "exit": "0.1.2", + "glob": "^7.1.1" + }, + "engines": { + "node": ">=0.2.5" + } + }, + "node_modules/cli-boxes": { + "version": "2.2.1", + "resolved": "https://registry.npmjs.org/cli-boxes/-/cli-boxes-2.2.1.tgz", + "integrity": "sha512-y4coMcylgSCdVinjiDBuR8PCC2bLjyGTwEmPb9NHR/QaNU6EUOXcTY/s6VjGMD6ENSEaeQYHCY0GNGS5jfMwPw==", + "dev": true, + "engines": { + "node": ">=6" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/cli-table": { + "version": "0.3.11", + "resolved": "https://registry.npmjs.org/cli-table/-/cli-table-0.3.11.tgz", + "integrity": "sha512-IqLQi4lO0nIB4tcdTpN4LCB9FI3uqrJZK7RC515EnhZ6qBaglkIgICb1wjeAqpdoOabm1+SuQtkXIPdYC93jhQ==", + "dependencies": { + "colors": "1.0.3" + }, + "engines": { + "node": ">= 0.2.0" + } + }, + "node_modules/cli-table/node_modules/colors": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/colors/-/colors-1.0.3.tgz", + "integrity": "sha512-pFGrxThWcWQ2MsAz6RtgeWe4NK2kUE1WfsrvvlctdII745EW9I0yflqhe7++M5LEc7bV2c/9/5zc8sFcpL0Drw==", + "engines": { + "node": ">=0.1.90" + } + }, + "node_modules/cliui": { + "version": "7.0.4", + "resolved": "https://registry.npmjs.org/cliui/-/cliui-7.0.4.tgz", + "integrity": "sha512-OcRE68cOsVMXp1Yvonl/fzkQOyjLSu/8bhPDfQt0e0/Eb283TKP20Fs2MqoPsr9SwA595rRCA+QMzYc9nBP+JQ==", + "dev": true, + "dependencies": { + "string-width": "^4.2.0", + "strip-ansi": "^6.0.0", + "wrap-ansi": "^7.0.0" + } + }, + "node_modules/clone": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/clone/-/clone-2.1.2.tgz", + "integrity": "sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w==", + "engines": { + "node": ">=0.8" + } + }, + "node_modules/clone-response": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/clone-response/-/clone-response-1.0.3.tgz", + "integrity": "sha512-ROoL94jJH2dUVML2Y/5PEDNaSHgeOdSDicUyS7izcF63G6sTc/FTjLub4b8Il9S8S0beOfYt0TaA5qvFK+w0wA==", + "dependencies": { + "mimic-response": "^1.0.0" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/collect-all": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/collect-all/-/collect-all-1.0.4.tgz", + "integrity": "sha512-RKZhRwJtJEP5FWul+gkSMEnaK6H3AGPTTWOiRimCcs+rc/OmQE3Yhy1Q7A7KsdkG3ZXVdZq68Y6ONSdvkeEcKA==", + "dev": true, + "dependencies": { + "stream-connect": "^1.0.2", + "stream-via": "^1.0.4" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/color-convert": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz", + "integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==", + "dev": true, + "dependencies": { + "color-name": "~1.1.4" + }, + "engines": { + "node": ">=7.0.0" + } + }, + "node_modules/color-name": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz", + "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==", + "dev": true + }, + "node_modules/color-support": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/color-support/-/color-support-1.1.3.tgz", + "integrity": "sha512-qiBjkpbMLO/HL68y+lh4q0/O1MZFj2RX6X/KmMa3+gJD3z+WwI1ZzDHysvqHGS3mP6mznPckpXmw1nI9cJjyRg==", + "optional": true, + "bin": { + "color-support": "bin.js" + } + }, + "node_modules/colors": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/colors/-/colors-1.1.2.tgz", + "integrity": "sha512-ENwblkFQpqqia6b++zLD/KUWafYlVY/UNnAp7oz7LY7E924wmpye416wBOmvv/HMWzl8gL1kJlfvId/1Dg176w==", + "dev": true, + "engines": { + "node": ">=0.1.90" + } + }, + "node_modules/combined-stream": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", + "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", + "dependencies": { + "delayed-stream": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/command-line-args": { + "version": "5.2.1", + "resolved": "https://registry.npmjs.org/command-line-args/-/command-line-args-5.2.1.tgz", + "integrity": "sha512-H4UfQhZyakIjC74I9d34fGYDwk3XpSr17QhEd0Q3I9Xq1CETHo4Hcuo87WyWHpAF1aSLjLRf5lD9ZGX2qStUvg==", + "dev": true, + "dependencies": { + "array-back": "^3.1.0", + "find-replace": "^3.0.0", + "lodash.camelcase": "^4.3.0", + "typical": "^4.0.0" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/command-line-args/node_modules/array-back": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-3.1.0.tgz", + "integrity": "sha512-TkuxA4UCOvxuDK6NZYXCalszEzj+TLszyASooky+i742l9TqsOdYCMJJupxRic61hwquNtppB3hgcuq9SVSH1Q==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/command-line-args/node_modules/typical": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/typical/-/typical-4.0.0.tgz", + "integrity": "sha512-VAH4IvQ7BDFYglMd7BPRDfLgxZZX4O4TFcRDA6EN5X7erNJJq+McIEp8np9aVtxrCJ6qx4GTYVfOWNjcqwZgRw==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/command-line-tool": { + "version": "0.8.0", + "resolved": "https://registry.npmjs.org/command-line-tool/-/command-line-tool-0.8.0.tgz", + "integrity": "sha512-Xw18HVx/QzQV3Sc5k1vy3kgtOeGmsKIqwtFFoyjI4bbcpSgnw2CWVULvtakyw4s6fhyAdI6soQQhXc2OzJy62g==", + "dev": true, + "dependencies": { + "ansi-escape-sequences": "^4.0.0", + "array-back": "^2.0.0", + "command-line-args": "^5.0.0", + "command-line-usage": "^4.1.0", + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/command-line-tool/node_modules/array-back": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-2.0.0.tgz", + "integrity": "sha512-eJv4pLLufP3g5kcZry0j6WXpIbzYw9GUB4mVJZno9wfwiBxbizTnHCw3VJb07cBihbFX48Y7oSrW9y+gt4glyw==", + "dev": true, + "dependencies": { + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/command-line-usage": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/command-line-usage/-/command-line-usage-4.1.0.tgz", + "integrity": "sha512-MxS8Ad995KpdAC0Jopo/ovGIroV/m0KHwzKfXxKag6FHOkGsH8/lv5yjgablcRxCJJC0oJeUMuO/gmaq+Wq46g==", + "dev": true, + "dependencies": { + "ansi-escape-sequences": "^4.0.0", + "array-back": "^2.0.0", + "table-layout": "^0.4.2", + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/command-line-usage/node_modules/array-back": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-2.0.0.tgz", + "integrity": "sha512-eJv4pLLufP3g5kcZry0j6WXpIbzYw9GUB4mVJZno9wfwiBxbizTnHCw3VJb07cBihbFX48Y7oSrW9y+gt4glyw==", + "dev": true, + "dependencies": { + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/commander": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/commander/-/commander-4.0.1.tgz", + "integrity": "sha512-IPF4ouhCP+qdlcmCedhxX4xiGBPyigb8v5NeUp+0LyhwLgxMqyp3S0vl7TAPfS/hiP7FC3caI/PB9lTmP8r1NA==", + "dev": true, + "engines": { + "node": ">= 6" + } + }, + "node_modules/commist": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/commist/-/commist-1.1.0.tgz", + "integrity": "sha512-rraC8NXWOEjhADbZe9QBNzLAN5Q3fsTPQtBV+fEVj6xKIgDgNiEVE6ZNfHpZOqfQ21YUzfVNUXLOEZquYvQPPg==", + "dependencies": { + "leven": "^2.1.0", + "minimist": "^1.1.0" + } + }, + "node_modules/common-sequence": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/common-sequence/-/common-sequence-2.0.2.tgz", + "integrity": "sha512-jAg09gkdkrDO9EWTdXfv80WWH3yeZl5oT69fGfedBNS9pXUKYInVJ1bJ+/ht2+Moeei48TmSbQDYMc8EOx9G0g==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/commondir": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/commondir/-/commondir-1.0.1.tgz", + "integrity": "sha512-W9pAhw0ja1Edb5GVdIF1mjZw/ASI0AlShXM83UUGe2DVr5TdAPEA1OA8m/g8zWp9x6On7gqufY+FatDbC3MDQg==", + "dev": true + }, + "node_modules/component-emitter": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.3.0.tgz", + "integrity": "sha512-Rd3se6QB+sO1TwqZjscQrurpEPIfO0/yYnSin6Q/rD3mOutHvUrCAhJub3r90uNb+SESBuE0QYoB90YdfatsRg==", + "dev": true + }, + "node_modules/compress-commons": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/compress-commons/-/compress-commons-4.1.1.tgz", + "integrity": "sha512-QLdDLCKNV2dtoTorqgxngQCMA+gWXkM/Nwu7FpeBhk/RdkzimqC3jueb/FDmaZeXh+uby1jkBqE3xArsLBE5wQ==", + "dev": true, + "dependencies": { + "buffer-crc32": "^0.2.13", + "crc32-stream": "^4.0.2", + "normalize-path": "^3.0.0", + "readable-stream": "^3.6.0" + }, + "engines": { + "node": ">= 10" + } + }, + "node_modules/concat-map": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz", + "integrity": "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg==" + }, + "node_modules/concat-stream": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/concat-stream/-/concat-stream-2.0.0.tgz", + "integrity": "sha512-MWufYdFw53ccGjCA+Ol7XJYpAlW6/prSMzuPOTRnJGcGzuhLn4Scrz7qf6o8bROZ514ltazcIFJZevcfbo0x7A==", + "engines": [ + "node >= 6.0" + ], + "dependencies": { + "buffer-from": "^1.0.0", + "inherits": "^2.0.3", + "readable-stream": "^3.0.2", + "typedarray": "^0.0.6" + } + }, + "node_modules/config-master": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/config-master/-/config-master-3.1.0.tgz", + "integrity": "sha512-n7LBL1zBzYdTpF1mx5DNcZnZn05CWIdsdvtPL4MosvqbBUK3Rq6VWEtGUuF3Y0s9/CIhMejezqlSkP6TnCJ/9g==", + "dev": true, + "dependencies": { + "walk-back": "^2.0.1" + } + }, + "node_modules/config-master/node_modules/walk-back": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/walk-back/-/walk-back-2.0.1.tgz", + "integrity": "sha512-Nb6GvBR8UWX1D+Le+xUq0+Q1kFmRBIWVrfLnQAOmcpEzA9oAxwJ9gIr36t9TWYfzvWRvuMtjHiVsJYEkXWaTAQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/configstore": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/configstore/-/configstore-5.0.1.tgz", + "integrity": "sha512-aMKprgk5YhBNyH25hj8wGt2+D52Sw1DRRIzqBwLp2Ya9mFmY8KPvvtvmna8SxVR9JMZ4kzMD68N22vlaRpkeFA==", + "dev": true, + "dependencies": { + "dot-prop": "^5.2.0", + "graceful-fs": "^4.1.2", + "make-dir": "^3.0.0", + "unique-string": "^2.0.0", + "write-file-atomic": "^3.0.0", + "xdg-basedir": "^4.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/configstore/node_modules/write-file-atomic": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/write-file-atomic/-/write-file-atomic-3.0.3.tgz", + "integrity": "sha512-AvHcyZ5JnSfq3ioSyjrBkH9yW4m7Ayk8/9My/DD9onKeu/94fwrMocemO2QAJFAlnnDN+ZDS+ZjAR5ua1/PV/Q==", + "dev": true, + "dependencies": { + "imurmurhash": "^0.1.4", + "is-typedarray": "^1.0.0", + "signal-exit": "^3.0.2", + "typedarray-to-buffer": "^3.1.5" + } + }, + "node_modules/console-browserify": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/console-browserify/-/console-browserify-1.1.0.tgz", + "integrity": "sha512-duS7VP5pvfsNLDvL1O4VOEbw37AI3A4ZUQYemvDlnpGrNu9tprR7BYWpDYwC0Xia0Zxz5ZupdiIrUp0GH1aXfg==", + "dev": true, + "dependencies": { + "date-now": "^0.1.4" + } + }, + "node_modules/console-control-strings": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/console-control-strings/-/console-control-strings-1.1.0.tgz", + "integrity": "sha512-ty/fTekppD2fIwRvnZAVdeOiGd1c7YXEixbgJTNzqcxJWKQnjJ/V1bNEEE6hygpM3WjwHFUVK6HTjWSzV4a8sQ==", + "optional": true + }, + "node_modules/content-disposition": { + "version": "0.5.4", + "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz", + "integrity": "sha512-FveZTNuGw04cxlAiWbzi6zTAL/lhehaWbTtgluJh4/E95DqMwTmha3KZN1aAWA8cFIhHzMZUvLevkw5Rqk+tSQ==", + "dependencies": { + "safe-buffer": "5.2.1" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/content-disposition/node_modules/safe-buffer": { + "version": "5.2.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", + "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ] + }, + "node_modules/content-type": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.4.tgz", + "integrity": "sha512-hIP3EEPs8tB9AT1L+NUqtwOAps4mk2Zob89MWXMHjHWg9milF/j4osnnQLXBCBFBk/tvIG/tUc9mOUJiPBhPXA==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/continuable-cache": { + "version": "0.3.1", + "resolved": "https://registry.npmjs.org/continuable-cache/-/continuable-cache-0.3.1.tgz", + "integrity": "sha512-TF30kpKhTH8AGCG3dut0rdd/19B7Z+qCnrMoBLpyQu/2drZdNrrpcjPEoJeSVsQM+8KmWG5O56oPDjSSUsuTyA==", + "dev": true + }, + "node_modules/convert-source-map": { + "version": "1.9.0", + "resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-1.9.0.tgz", + "integrity": "sha512-ASFBup0Mz1uyiIjANan1jzLQami9z1PoYSZCiiYW2FczPbenXc45FZdBZLzOT+r6+iciuEModtmCti+hjaAk0A==", + "dev": true + }, + "node_modules/cookie": { + "version": "0.4.2", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.4.2.tgz", + "integrity": "sha512-aSWTXFzaKWkvHO1Ny/s+ePFpvKsPnjc551iI41v3ny/ow6tBG5Vd+FuqGNhh1LxOmVzOlGUriIlOaokOvhaStA==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/cookie-parser": { + "version": "1.4.6", + "resolved": "https://registry.npmjs.org/cookie-parser/-/cookie-parser-1.4.6.tgz", + "integrity": "sha512-z3IzaNjdwUC2olLIB5/ITd0/setiaFMLYiZJle7xg5Fe9KWAceil7xszYfHHBtDFYLSgJduS2Ty0P1uJdPDJeA==", + "dependencies": { + "cookie": "0.4.1", + "cookie-signature": "1.0.6" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/cookie-parser/node_modules/cookie": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.4.1.tgz", + "integrity": "sha512-ZwrFkGJxUR3EIoXtO+yVE69Eb7KlixbaeAWfBQB9vVsNn/o+Yw69gBWSSDK825hQNdN+wF8zELf3dFNl/kxkUA==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/cookie-signature": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz", + "integrity": "sha512-QADzlaHc8icV8I7vbaJXJwod9HWYp8uCqf1xa4OfNu1T7JVxQIrUgOWtHdNDtPiywmFbiS12VjotIXLrKM3orQ==" + }, + "node_modules/cookiejar": { + "version": "2.1.4", + "resolved": "https://registry.npmjs.org/cookiejar/-/cookiejar-2.1.4.tgz", + "integrity": "sha512-LDx6oHrK+PhzLKJU9j5S7/Y3jM/mUHvD/DeI1WQmJn652iPC5Y4TBzC9l+5OMOXlyTTA+SmVUPm0HQUwpD5Jqw==", + "dev": true + }, + "node_modules/core-util-is": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.3.tgz", + "integrity": "sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ==" + }, + "node_modules/cors": { + "version": "2.8.5", + "resolved": "https://registry.npmjs.org/cors/-/cors-2.8.5.tgz", + "integrity": "sha512-KIHbLJqu73RGr/hnbrO9uBeixNGuvSQjul/jdFvS/KFSIH1hWVd1ng7zOHx+YrEfInLG7q4n6GHQ9cDtxv/P6g==", + "dependencies": { + "object-assign": "^4", + "vary": "^1" + }, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/cp-file": { + "version": "6.2.0", + "resolved": "https://registry.npmjs.org/cp-file/-/cp-file-6.2.0.tgz", + "integrity": "sha512-fmvV4caBnofhPe8kOcitBwSn2f39QLjnAnGq3gO9dfd75mUytzKNZB1hde6QHunW2Rt+OwuBOMc3i1tNElbszA==", + "dev": true, + "dependencies": { + "graceful-fs": "^4.1.2", + "make-dir": "^2.0.0", + "nested-error-stacks": "^2.0.0", + "pify": "^4.0.1", + "safe-buffer": "^5.0.1" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/cp-file/node_modules/make-dir": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/make-dir/-/make-dir-2.1.0.tgz", + "integrity": "sha512-LS9X+dc8KLxXCb8dni79fLIIUA5VyZoyjSMCwTluaXA0o27cCK0bhXkpgw+sTXVpPy/lSO57ilRixqk0vDmtRA==", + "dev": true, + "dependencies": { + "pify": "^4.0.1", + "semver": "^5.6.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/cp-file/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/crc-32": { + "version": "1.2.2", + "resolved": "https://registry.npmjs.org/crc-32/-/crc-32-1.2.2.tgz", + "integrity": "sha512-ROmzCKrTnOwybPcJApAA6WBWij23HVfGVNKqqrZpuyZOHqK2CwHSvpGuyt/UNNvaIjEd8X5IFGp4Mh+Ie1IHJQ==", + "dev": true, + "bin": { + "crc32": "bin/crc32.njs" + }, + "engines": { + "node": ">=0.8" + } + }, + "node_modules/crc32-stream": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/crc32-stream/-/crc32-stream-4.0.2.tgz", + "integrity": "sha512-DxFZ/Hk473b/muq1VJ///PMNLj0ZMnzye9thBpmjpJKCc5eMgB95aK8zCGrGfQ90cWo561Te6HK9D+j4KPdM6w==", + "dev": true, + "dependencies": { + "crc-32": "^1.2.0", + "readable-stream": "^3.4.0" + }, + "engines": { + "node": ">= 10" + } + }, + "node_modules/cronosjs": { + "version": "1.7.1", + "resolved": "https://registry.npmjs.org/cronosjs/-/cronosjs-1.7.1.tgz", + "integrity": "sha512-d6S6+ep7dJxsAG8OQQCdKuByI/S/AV64d9OF5mtmcykOyPu92cAkAnF3Tbc9s5oOaLQBYYQmTNvjqYRkPJ/u5Q==", + "engines": { + "node": ">=8.0.0" + } + }, + "node_modules/cross-fetch": { + "version": "3.1.5", + "resolved": "https://registry.npmjs.org/cross-fetch/-/cross-fetch-3.1.5.tgz", + "integrity": "sha512-lvb1SBsI0Z7GDwmuid+mU3kWVBwTVUbe7S0H52yaaAdQOXq2YktTCZdlAcNKFzE6QtRz0snpw9bNiPeOIkkQvw==", + "dev": true, + "dependencies": { + "node-fetch": "2.6.7" + } + }, + "node_modules/cross-fetch/node_modules/node-fetch": { + "version": "2.6.7", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.7.tgz", + "integrity": "sha512-ZjMPFEfVx5j+y2yF35Kzx5sF7kDzxuDj6ziH4FFbOp87zKDZNx8yExJIb05OGF4Nlt9IHFIMBkRl41VdvcNdbQ==", + "dev": true, + "dependencies": { + "whatwg-url": "^5.0.0" + }, + "engines": { + "node": "4.x || >=6.0.0" + }, + "peerDependencies": { + "encoding": "^0.1.0" + }, + "peerDependenciesMeta": { + "encoding": { + "optional": true + } + } + }, + "node_modules/cross-spawn": { + "version": "7.0.3", + "resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.3.tgz", + "integrity": "sha512-iRDPJKUPVEND7dHPO8rkbOnPpyDygcDFtWjpeWNCgy8WP2rXcxXL8TskReQl6OrB2G7+UJrags1q15Fudc7G6w==", + "dev": true, + "dependencies": { + "path-key": "^3.1.0", + "shebang-command": "^2.0.0", + "which": "^2.0.1" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/crypto-random-string": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/crypto-random-string/-/crypto-random-string-2.0.0.tgz", + "integrity": "sha512-v1plID3y9r/lPhviJ1wrXpLeyUIGAZ2SHNYTEapm7/8A9nLPoyvVp3RK/EPFqn5kEznyWgYZNsRtYYIWbuG8KA==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/css-select": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/css-select/-/css-select-4.3.0.tgz", + "integrity": "sha512-wPpOYtnsVontu2mODhA19JrqWxNsfdatRKd64kmpRbQgh1KtItko5sTnEpPdpSaJszTOhEMlF/RPz28qj4HqhQ==", + "dependencies": { + "boolbase": "^1.0.0", + "css-what": "^6.0.1", + "domhandler": "^4.3.1", + "domutils": "^2.8.0", + "nth-check": "^2.0.1" + }, + "funding": { + "url": "https://github.com/sponsors/fb55" + } + }, + "node_modules/css-what": { + "version": "6.1.0", + "resolved": "https://registry.npmjs.org/css-what/-/css-what-6.1.0.tgz", + "integrity": "sha512-HTUrgRJ7r4dsZKU6GjmpfRK1O76h97Z8MfS1G0FozR+oF2kG6Vfe8JE6zwrkbxigziPHinCJ+gCPjA9EaBDtRw==", + "engines": { + "node": ">= 6" + }, + "funding": { + "url": "https://github.com/sponsors/fb55" + } + }, + "node_modules/date-now": { + "version": "0.1.4", + "resolved": "https://registry.npmjs.org/date-now/-/date-now-0.1.4.tgz", + "integrity": "sha512-AsElvov3LoNB7tf5k37H2jYSB+ZZPMT5sG2QjJCcdlV5chIv6htBUBUui2IKRjgtKAKtCBN7Zbwa+MtwLjSeNw==", + "dev": true + }, + "node_modules/dateformat": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/dateformat/-/dateformat-3.0.3.tgz", + "integrity": "sha512-jyCETtSl3VMZMWeRo7iY1FL19ges1t55hMo5yaam4Jrsm5EPL89UQkoQRyiI+Yf4k8r2ZpdngkV8hr1lIdjb3Q==", + "dev": true, + "engines": { + "node": "*" + } + }, + "node_modules/debug": { + "version": "2.6.9", + "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz", + "integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==", + "dependencies": { + "ms": "2.0.0" + } + }, + "node_modules/decamelize": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/decamelize/-/decamelize-4.0.0.tgz", + "integrity": "sha512-9iE1PgSik9HeIIw2JO94IidnE3eBoQrFJ3w7sFuzSX4DpmZ3v5sZpUiV5Swcf6mQEF+Y0ru8Neo+p+nyh2J+hQ==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/decompress-response": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz", + "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==", + "dependencies": { + "mimic-response": "^3.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/decompress-response/node_modules/mimic-response": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz", + "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/deep-extend": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz", + "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==", + "dev": true, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/default-require-extensions": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/default-require-extensions/-/default-require-extensions-2.0.0.tgz", + "integrity": "sha512-B0n2zDIXpzLzKeoEozorDSa1cHc1t0NjmxP0zuAxbizNU2MBqYJJKYXrrFdKuQliojXynrxgd7l4ahfg/+aA5g==", + "dev": true, + "dependencies": { + "strip-bom": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/defer-to-connect": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/defer-to-connect/-/defer-to-connect-2.0.1.tgz", + "integrity": "sha512-4tvttepXG1VaYGrRibk5EwJd1t4udunSOVMdLSAL6mId1ix438oPwPZMALY41FCijukO1L0twNcGsdzS7dHgDg==", + "engines": { + "node": ">=10" + } + }, + "node_modules/delayed-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", + "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==", + "engines": { + "node": ">=0.4.0" + } + }, + "node_modules/delegates": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delegates/-/delegates-1.0.0.tgz", + "integrity": "sha512-bd2L678uiWATM6m5Z1VzNCErI3jiGzt6HGY8OVICs40JQq/HALfbyNJmp0UDakEY4pMMaN0Ly5om/B1VI/+xfQ==", + "optional": true + }, + "node_modules/denque": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/denque/-/denque-2.0.1.tgz", + "integrity": "sha512-tfiWc6BQLXNLpNiR5iGd0Ocu3P3VpxfzFiqubLgMfhfOw9WyvgJBd46CClNn9k3qfbjvT//0cf7AlYRX/OslMQ==", + "engines": { + "node": ">=0.10" + } + }, + "node_modules/depd": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/depd/-/depd-1.1.2.tgz", + "integrity": "sha512-7emPTl6Dpo6JRXOXjLRxck+FlLRX5847cLKEn00PLAgc3g2hTZZgr+e4c2v6QpSmLeFP3n5yUo7ft6avBK/5jQ==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/destroy": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz", + "integrity": "sha512-3NdhDuEXnfun/z7x9GOElY49LoqVHoGScmOKwmxhsS8N5Y+Z8KyPPDnaSzqWgYt/ji4mqwfTS34Htrk0zPIXVg==" + }, + "node_modules/detect-file": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/detect-file/-/detect-file-1.0.0.tgz", + "integrity": "sha512-DtCOLG98P007x7wiiOmfI0fi3eIKyWiLTGJ2MDnVi/E04lWGbf+JzrRHMm0rgIIZJGtHpKpbVgLWHrv8xXpc3Q==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/detect-libc": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.0.1.tgz", + "integrity": "sha512-463v3ZeIrcWtdgIg6vI6XUncguvr2TnGl4SzDXinkt9mSLpBJKXT3mW6xT3VQdDN11+WVs29pgvivTc4Lp8v+w==", + "optional": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/dezalgo": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/dezalgo/-/dezalgo-1.0.4.tgz", + "integrity": "sha512-rXSP0bf+5n0Qonsb+SVVfNfIsimO4HEtmnIpPHY8Q1UCzKlQrDMfdobr8nJOOsRgWCyMRqeSBQzmWUMq7zvVig==", + "dev": true, + "dependencies": { + "asap": "^2.0.0", + "wrappy": "1" + } + }, + "node_modules/dicer": { + "version": "0.2.5", + "resolved": "https://registry.npmjs.org/dicer/-/dicer-0.2.5.tgz", + "integrity": "sha512-FDvbtnq7dzlPz0wyYlOExifDEZcu8h+rErEXgfxqmLfRfC/kJidEFh4+effJRO3P0xmfqyPbSMG0LveNRfTKVg==", + "dependencies": { + "readable-stream": "1.1.x", + "streamsearch": "0.1.2" + }, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/dicer/node_modules/isarray": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", + "integrity": "sha512-D2S+3GLxWH+uhrNEcoh/fnmYeP8E8/zHl644d/jdA0g2uyXvy3sb0qxotE+ne0LtccHknQzWwZEzhak7oJ0COQ==" + }, + "node_modules/dicer/node_modules/readable-stream": { + "version": "1.1.14", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz", + "integrity": "sha512-+MeVjFf4L44XUkhM1eYbD8fyEsxcV81pqMSR5gblfcLCHfZvbrqy4/qYHE+/R5HoBUT11WV5O08Cr1n3YXkWVQ==", + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.1", + "isarray": "0.0.1", + "string_decoder": "~0.10.x" + } + }, + "node_modules/dicer/node_modules/string_decoder": { + "version": "0.10.31", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz", + "integrity": "sha512-ev2QzSzWPYmy9GuqfIVildA4OdcGLeFZQrq5ys6RtiuF+RQQiZWr8TZNyAcuVXyQRYfEO+MsoB/1BuQVhOJuoQ==" + }, + "node_modules/diff": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/diff/-/diff-5.0.0.tgz", + "integrity": "sha512-/VTCrvm5Z0JGty/BWHljh+BAiw3IK+2j87NGMu8Nwc/f48WoDAC395uomO9ZD117ZOBaHmkX1oyLvkVM/aIT3w==", + "dev": true, + "engines": { + "node": ">=0.3.1" + } + }, + "node_modules/dmd": { + "version": "6.2.0", + "resolved": "https://registry.npmjs.org/dmd/-/dmd-6.2.0.tgz", + "integrity": "sha512-uXWxLF1H7TkUAuoHK59/h/ts5cKavm2LnhrIgJWisip4BVzPoXavlwyoprFFn2CzcahKYgvkfaebS6oxzgflkg==", + "dev": true, + "dependencies": { + "array-back": "^6.2.2", + "cache-point": "^2.0.0", + "common-sequence": "^2.0.2", + "file-set": "^4.0.2", + "handlebars": "^4.7.7", + "marked": "^4.2.3", + "object-get": "^2.1.1", + "reduce-flatten": "^3.0.1", + "reduce-unique": "^2.0.1", + "reduce-without": "^1.0.1", + "test-value": "^3.0.0", + "walk-back": "^5.1.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/dmd/node_modules/marked": { + "version": "4.2.12", + "resolved": "https://registry.npmjs.org/marked/-/marked-4.2.12.tgz", + "integrity": "sha512-yr8hSKa3Fv4D3jdZmtMMPghgVt6TWbk86WQaWhDloQjRSQhMMYCAro7jP7VDJrjjdV8pxVxMssXS8B8Y5DZ5aw==", + "dev": true, + "bin": { + "marked": "bin/marked.js" + }, + "engines": { + "node": ">= 12" + } + }, + "node_modules/dom-serializer": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-1.4.1.tgz", + "integrity": "sha512-VHwB3KfrcOOkelEG2ZOfxqLZdfkil8PtJi4P8N2MMXucZq2yLp75ClViUlOVwyoHEDjYU433Aq+5zWP61+RGag==", + "dependencies": { + "domelementtype": "^2.0.1", + "domhandler": "^4.2.0", + "entities": "^2.0.0" + }, + "funding": { + "url": "https://github.com/cheeriojs/dom-serializer?sponsor=1" + } + }, + "node_modules/domelementtype": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz", + "integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fb55" + } + ] + }, + "node_modules/domhandler": { + "version": "4.3.1", + "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-4.3.1.tgz", + "integrity": "sha512-GrwoxYN+uWlzO8uhUXRl0P+kHE4GtVPfYzVLcUxPL7KNdHKj66vvlhiweIHqYYXWlw+T8iLMp42Lm67ghw4WMQ==", + "dependencies": { + "domelementtype": "^2.2.0" + }, + "engines": { + "node": ">= 4" + }, + "funding": { + "url": "https://github.com/fb55/domhandler?sponsor=1" + } + }, + "node_modules/dompurify": { + "version": "2.3.6", + "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-2.3.6.tgz", + "integrity": "sha512-OFP2u/3T1R5CEgWCEONuJ1a5+MFKnOYpkywpUSxv/dj1LeBT1erK+JwM7zK0ROy2BRhqVCf0LRw/kHqKuMkVGg==", + "dev": true + }, + "node_modules/domutils": { + "version": "2.8.0", + "resolved": "https://registry.npmjs.org/domutils/-/domutils-2.8.0.tgz", + "integrity": "sha512-w96Cjofp72M5IIhpjgobBimYEfoPjx1Vx0BSX9P30WBdZW2WIKU0T1Bd0kz2eNZ9ikjKgHbEyKx8BB6H1L3h3A==", + "dependencies": { + "dom-serializer": "^1.0.1", + "domelementtype": "^2.2.0", + "domhandler": "^4.2.0" + }, + "funding": { + "url": "https://github.com/fb55/domutils?sponsor=1" + } + }, + "node_modules/dot-prop": { + "version": "5.3.0", + "resolved": "https://registry.npmjs.org/dot-prop/-/dot-prop-5.3.0.tgz", + "integrity": "sha512-QM8q3zDe58hqUqjraQOmzZ1LIH9SWQJTlEKCH4kJ2oQvLZk7RbQXvtDM2XEq3fwkV9CCvvH4LA0AV+ogFsBM2Q==", + "dev": true, + "dependencies": { + "is-obj": "^2.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/duplexer": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/duplexer/-/duplexer-0.1.2.tgz", + "integrity": "sha512-jtD6YG370ZCIi/9GTaJKQxWTZD045+4R4hTk/x1UyoqadyJ9x9CgSi1RlVDQF8U2sxLLSnFkCaMihqljHIWgMg==", + "dev": true + }, + "node_modules/duplexer3": { + "version": "0.1.5", + "resolved": "https://registry.npmjs.org/duplexer3/-/duplexer3-0.1.5.tgz", + "integrity": "sha512-1A8za6ws41LQgv9HrE/66jyC5yuSjQ3L/KOpFtoBilsAK2iA2wuS5rTt1OCzIvtS2V7nVmedsUU+DGRcjBmOYA==", + "dev": true + }, + "node_modules/duplexify": { + "version": "4.1.2", + "resolved": "https://registry.npmjs.org/duplexify/-/duplexify-4.1.2.tgz", + "integrity": "sha512-fz3OjcNCHmRP12MJoZMPglx8m4rrFP8rovnk4vT8Fs+aonZoCwGg10dSsQsfP/E62eZcPTMSMP6686fu9Qlqtw==", + "dependencies": { + "end-of-stream": "^1.4.1", + "inherits": "^2.0.3", + "readable-stream": "^3.1.1", + "stream-shift": "^1.0.0" + } + }, + "node_modules/ee-first": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz", + "integrity": "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==" + }, + "node_modules/emoji-regex": { + "version": "8.0.0", + "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz", + "integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==", + "devOptional": true + }, + "node_modules/encodeurl": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.2.tgz", + "integrity": "sha512-TPJXq8JqFaVYm2CWmPvnP2Iyo4ZSM7/QKcSmuMLDObfpH5fi7RUGmd/rTDf+rut/saiDiQEeVTNgAmJEdAOx0w==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/end-of-stream": { + "version": "1.4.4", + "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.4.tgz", + "integrity": "sha512-+uw1inIHVPQoaVuHzRyXd21icM+cnt4CzD5rW+NC1wjOUSTOs+Te7FOv7AhN7vS9x/oIyhLP5PR1H+phQAHu5Q==", + "dependencies": { + "once": "^1.4.0" + } + }, + "node_modules/enquirer": { + "version": "2.3.6", + "resolved": "https://registry.npmjs.org/enquirer/-/enquirer-2.3.6.tgz", + "integrity": "sha512-yjNnPr315/FjS4zIsUxYguYUPP2e1NK4d7E7ZOLiyYCcbFBiTMyID+2wvm2w6+pZ/odMA7cRkjhsPbltwBOrLg==", + "dependencies": { + "ansi-colors": "^4.1.1" + }, + "engines": { + "node": ">=8.6" + } + }, + "node_modules/entities": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/entities/-/entities-2.2.0.tgz", + "integrity": "sha512-p92if5Nz619I0w+akJrLZH0MX0Pb5DX39XOwQTtXSdQQOaYH03S1uIQp4mhOZtAXrxq4ViO67YTiLBo2638o9A==", + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, + "node_modules/error": { + "version": "7.2.1", + "resolved": "https://registry.npmjs.org/error/-/error-7.2.1.tgz", + "integrity": "sha512-fo9HBvWnx3NGUKMvMwB/CBCMMrfEJgbDTVDEkPygA3Bdd3lM1OyCd+rbQ8BwnpF6GdVeOLDNmyL4N5Bg80ZvdA==", + "dev": true, + "dependencies": { + "string-template": "~0.2.1" + } + }, + "node_modules/error-ex": { + "version": "1.3.2", + "resolved": "https://registry.npmjs.org/error-ex/-/error-ex-1.3.2.tgz", + "integrity": "sha512-7dFHNmqeFSEt2ZBsCriorKnn3Z2pj+fd9kmI6QoWw4//DL+icEBfc0U7qJCisqrTsKTjw4fNFy2pW9OqStD84g==", + "dev": true, + "dependencies": { + "is-arrayish": "^0.2.1" + } + }, + "node_modules/es6-error": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/es6-error/-/es6-error-4.1.1.tgz", + "integrity": "sha512-Um/+FxMr9CISWh0bi5Zv0iOD+4cFh5qLeks1qhAopKVAJw3drgKbKySikp7wGhDL0HPeaja0P5ULZrxLkniUVg==", + "dev": true + }, + "node_modules/escalade": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.1.1.tgz", + "integrity": "sha512-k0er2gUkLf8O0zKJiAhmkTnJlTvINGv7ygDNPbeIsX/TJjGJZHuh9B2UxbsaEkmlEo9MfhrSzmhIlhRlI2GXnw==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/escape-goat": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/escape-goat/-/escape-goat-2.1.1.tgz", + "integrity": "sha512-8/uIhbG12Csjy2JEW7D9pHbreaVaS/OpN3ycnyvElTdwM5n6GY6W6e2IPemfvGZeUMqZ9A/3GqIZMgKnBhAw/Q==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/escape-html": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz", + "integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==" + }, + "node_modules/escape-string-regexp": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-2.0.0.tgz", + "integrity": "sha512-UpzcLCXolUWcNu5HtVMHYdXJjArjsF9C0aNnquZYY4uW/Vu0miy5YoWvbV345HauVvcAUnpRuhMMcqTcGOY2+w==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/esprima": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz", + "integrity": "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A==", + "bin": { + "esparse": "bin/esparse.js", + "esvalidate": "bin/esvalidate.js" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/etag": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz", + "integrity": "sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/eventemitter2": { + "version": "0.4.14", + "resolved": "https://registry.npmjs.org/eventemitter2/-/eventemitter2-0.4.14.tgz", + "integrity": "sha512-K7J4xq5xAD5jHsGM5ReWXRTFa3JRGofHiMcVgQ8PRwgWxzjHpMWCIzsmyf60+mh8KLsqYPcjUMa0AC4hd6lPyQ==", + "dev": true + }, + "node_modules/exit": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/exit/-/exit-0.1.2.tgz", + "integrity": "sha512-Zk/eNKV2zbjpKzrsQ+n1G6poVbErQxJ0LBOJXaKZ1EViLzH+hrLu9cdXI4zw9dBQJslwBEpbQ2P1oS7nDxs6jQ==", + "dev": true, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/expand-tilde": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/expand-tilde/-/expand-tilde-2.0.2.tgz", + "integrity": "sha512-A5EmesHW6rfnZ9ysHQjPdJRni0SRar0tjtG5MNtm9n5TUvsYU8oozprtRD4AqHxcZWWlVuAmQo2nWKfN9oyjTw==", + "dev": true, + "dependencies": { + "homedir-polyfill": "^1.0.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/express": { + "version": "4.17.2", + "resolved": "https://registry.npmjs.org/express/-/express-4.17.2.tgz", + "integrity": "sha512-oxlxJxcQlYwqPWKVJJtvQiwHgosH/LrLSPA+H4UxpyvSS6jC5aH+5MoHFM+KABgTOt0APue4w66Ha8jCUo9QGg==", + "dependencies": { + "accepts": "~1.3.7", + "array-flatten": "1.1.1", + "body-parser": "1.19.1", + "content-disposition": "0.5.4", + "content-type": "~1.0.4", + "cookie": "0.4.1", + "cookie-signature": "1.0.6", + "debug": "2.6.9", + "depd": "~1.1.2", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "finalhandler": "~1.1.2", + "fresh": "0.5.2", + "merge-descriptors": "1.0.1", + "methods": "~1.1.2", + "on-finished": "~2.3.0", + "parseurl": "~1.3.3", + "path-to-regexp": "0.1.7", + "proxy-addr": "~2.0.7", + "qs": "6.9.6", + "range-parser": "~1.2.1", + "safe-buffer": "5.2.1", + "send": "0.17.2", + "serve-static": "1.14.2", + "setprototypeof": "1.2.0", + "statuses": "~1.5.0", + "type-is": "~1.6.18", + "utils-merge": "1.0.1", + "vary": "~1.1.2" + }, + "engines": { + "node": ">= 0.10.0" + } + }, + "node_modules/express-session": { + "version": "1.17.2", + "resolved": "https://registry.npmjs.org/express-session/-/express-session-1.17.2.tgz", + "integrity": "sha512-mPcYcLA0lvh7D4Oqr5aNJFMtBMKPLl++OKKxkHzZ0U0oDq1rpKBnkR5f5vCHR26VeArlTOEF9td4x5IjICksRQ==", + "dependencies": { + "cookie": "0.4.1", + "cookie-signature": "1.0.6", + "debug": "2.6.9", + "depd": "~2.0.0", + "on-headers": "~1.0.2", + "parseurl": "~1.3.3", + "safe-buffer": "5.2.1", + "uid-safe": "~2.1.5" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/express-session/node_modules/cookie": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.4.1.tgz", + "integrity": "sha512-ZwrFkGJxUR3EIoXtO+yVE69Eb7KlixbaeAWfBQB9vVsNn/o+Yw69gBWSSDK825hQNdN+wF8zELf3dFNl/kxkUA==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/express-session/node_modules/depd": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz", + "integrity": "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/express-session/node_modules/safe-buffer": { + "version": "5.2.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", + "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ] + }, + "node_modules/express/node_modules/cookie": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.4.1.tgz", + "integrity": "sha512-ZwrFkGJxUR3EIoXtO+yVE69Eb7KlixbaeAWfBQB9vVsNn/o+Yw69gBWSSDK825hQNdN+wF8zELf3dFNl/kxkUA==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/express/node_modules/safe-buffer": { + "version": "5.2.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", + "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ] + }, + "node_modules/extend": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz", + "integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==", + "dev": true + }, + "node_modules/fast-deep-equal": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", + "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==" + }, + "node_modules/fast-json-stable-stringify": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.1.0.tgz", + "integrity": "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==", + "dev": true + }, + "node_modules/fast-safe-stringify": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/fast-safe-stringify/-/fast-safe-stringify-2.1.1.tgz", + "integrity": "sha512-W+KJc2dmILlPplD/H4K9l9LcAHAfPtP6BY84uVLXQ6Evcz9Lcg33Y2z1IVblT6xdY54PXYVHEv+0Wpq8Io6zkA==", + "dev": true + }, + "node_modules/faye-websocket": { + "version": "0.10.0", + "resolved": "https://registry.npmjs.org/faye-websocket/-/faye-websocket-0.10.0.tgz", + "integrity": "sha512-Xhj93RXbMSq8urNCUq4p9l0P6hnySJ/7YNRhYNug0bLOuii7pKO7xQFb5mx9xZXWCar88pLPb805PvUkwrLZpQ==", + "dev": true, + "dependencies": { + "websocket-driver": ">=0.5.1" + }, + "engines": { + "node": ">=0.4.0" + } + }, + "node_modules/figures": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/figures/-/figures-1.7.0.tgz", + "integrity": "sha512-UxKlfCRuCBxSXU4C6t9scbDyWZ4VlaFFdojKtzJuSkuOBQ5CNFum+zZXFwHjo+CxBC1t6zlYPgHIgFjL8ggoEQ==", + "dev": true, + "dependencies": { + "escape-string-regexp": "^1.0.5", + "object-assign": "^4.1.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/figures/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/file-set": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/file-set/-/file-set-4.0.2.tgz", + "integrity": "sha512-fuxEgzk4L8waGXaAkd8cMr73Pm0FxOVkn8hztzUW7BAHhOGH90viQNXbiOsnecCWmfInqU6YmAMwxRMdKETceQ==", + "dev": true, + "dependencies": { + "array-back": "^5.0.0", + "glob": "^7.1.6" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/file-set/node_modules/array-back": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-5.0.0.tgz", + "integrity": "sha512-kgVWwJReZWmVuWOQKEOohXKJX+nD02JAZ54D1RRWlv8L0NebauKAaFxACKzB74RTclt1+WNz5KHaLRDAPZbDEw==", + "dev": true, + "engines": { + "node": ">=10" + } + }, + "node_modules/file-sync-cmp": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/file-sync-cmp/-/file-sync-cmp-0.1.1.tgz", + "integrity": "sha512-0k45oWBokCqh2MOexeYKpyqmGKG+8mQ2Wd8iawx+uWd/weWJQAZ6SoPybagdCI4xFisag8iAR77WPm4h3pTfxA==", + "dev": true + }, + "node_modules/fill-range": { + "version": "7.0.1", + "resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.0.1.tgz", + "integrity": "sha512-qOo9F+dMUmC2Lcb4BbVvnKJxTPjCm+RRpe4gDuGrzkL7mEVl/djYSu2OdQ2Pa302N4oqkSg9ir6jaLWJ2USVpQ==", + "dev": true, + "dependencies": { + "to-regex-range": "^5.0.1" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/finalhandler": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.1.2.tgz", + "integrity": "sha512-aAWcW57uxVNrQZqFXjITpW3sIUQmHGG3qSb9mUah9MgMC4NeWhNOlNjXEYq3HjRAvL6arUviZGGJsBg6z0zsWA==", + "dependencies": { + "debug": "2.6.9", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "on-finished": "~2.3.0", + "parseurl": "~1.3.3", + "statuses": "~1.5.0", + "unpipe": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/find-cache-dir": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/find-cache-dir/-/find-cache-dir-2.1.0.tgz", + "integrity": "sha512-Tq6PixE0w/VMFfCgbONnkiQIVol/JJL7nRMi20fqzA4NRs9AfeqMGeRdPi3wIhYkxjeBaWh2rxwapn5Tu3IqOQ==", + "dev": true, + "dependencies": { + "commondir": "^1.0.1", + "make-dir": "^2.0.0", + "pkg-dir": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/find-cache-dir/node_modules/make-dir": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/make-dir/-/make-dir-2.1.0.tgz", + "integrity": "sha512-LS9X+dc8KLxXCb8dni79fLIIUA5VyZoyjSMCwTluaXA0o27cCK0bhXkpgw+sTXVpPy/lSO57ilRixqk0vDmtRA==", + "dev": true, + "dependencies": { + "pify": "^4.0.1", + "semver": "^5.6.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/find-cache-dir/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/find-replace": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/find-replace/-/find-replace-3.0.0.tgz", + "integrity": "sha512-6Tb2myMioCAgv5kfvP5/PkZZ/ntTpVK39fHY7WkWBgvbeE+VHd/tZuZ4mrC+bxh4cfOZeYKVPaJIZtZXV7GNCQ==", + "dev": true, + "dependencies": { + "array-back": "^3.0.1" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/find-replace/node_modules/array-back": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-3.1.0.tgz", + "integrity": "sha512-TkuxA4UCOvxuDK6NZYXCalszEzj+TLszyASooky+i742l9TqsOdYCMJJupxRic61hwquNtppB3hgcuq9SVSH1Q==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/find-up": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/find-up/-/find-up-5.0.0.tgz", + "integrity": "sha512-78/PXT1wlLLDgTzDs7sjq9hzz0vXD+zn+7wypEe4fXQxCmdmqfGsEPQxmiCSQI3ajFV91bVSsvNtrJRiW6nGng==", + "dev": true, + "dependencies": { + "locate-path": "^6.0.0", + "path-exists": "^4.0.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/findup-sync": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/findup-sync/-/findup-sync-0.3.0.tgz", + "integrity": "sha512-z8Nrwhi6wzxNMIbxlrTzuUW6KWuKkogZ/7OdDVq+0+kxn77KUH1nipx8iU6suqkHqc4y6n7a9A8IpmxY/pTjWg==", + "dev": true, + "dependencies": { + "glob": "~5.0.0" + }, + "engines": { + "node": ">= 0.6.0" + } + }, + "node_modules/findup-sync/node_modules/glob": { + "version": "5.0.15", + "resolved": "https://registry.npmjs.org/glob/-/glob-5.0.15.tgz", + "integrity": "sha512-c9IPMazfRITpmAAKi22dK1VKxGDX9ehhqfABDriL/lzO92xcUKEJPQHrVA/2YHSNFB4iFlykVmWvwo48nr3OxA==", + "dev": true, + "dependencies": { + "inflight": "^1.0.4", + "inherits": "2", + "minimatch": "2 || 3", + "once": "^1.3.0", + "path-is-absolute": "^1.0.0" + }, + "engines": { + "node": "*" + } + }, + "node_modules/fined": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/fined/-/fined-1.2.0.tgz", + "integrity": "sha512-ZYDqPLGxDkDhDZBjZBb+oD1+j0rA4E0pXY50eplAAOPg2N/gUBSSk5IM1/QhPfyVo19lJ+CvXpqfvk+b2p/8Ng==", + "dev": true, + "dependencies": { + "expand-tilde": "^2.0.2", + "is-plain-object": "^2.0.3", + "object.defaults": "^1.1.0", + "object.pick": "^1.2.0", + "parse-filepath": "^1.0.1" + }, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/flagged-respawn": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/flagged-respawn/-/flagged-respawn-1.0.1.tgz", + "integrity": "sha512-lNaHNVymajmk0OJMBn8fVUAU1BtDeKIqKoVhk4xAALB57aALg6b4W0MfJ/cUE0g9YBXy5XhSlPIpYIJ7HaY/3Q==", + "dev": true, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/flat": { + "version": "5.0.2", + "resolved": "https://registry.npmjs.org/flat/-/flat-5.0.2.tgz", + "integrity": "sha512-b6suED+5/3rTpUBdG1gupIl8MPFCAMA0QXwmljLhvCUKcUvdE4gWky9zpuGCcXHOsz4J9wPGNWq6OKpmIzz3hQ==", + "dev": true, + "bin": { + "flat": "cli.js" + } + }, + "node_modules/follow-redirects": { + "version": "1.15.2", + "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.2.tgz", + "integrity": "sha512-VQLG33o04KaQ8uYi2tVNbdrWp1QWxNNea+nmIB4EVM28v0hmP17z7aG1+wAkNzVq4KeXTq3221ye5qTJP91JwA==", + "funding": [ + { + "type": "individual", + "url": "https://github.com/sponsors/RubenVerborgh" + } + ], + "engines": { + "node": ">=4.0" + }, + "peerDependenciesMeta": { + "debug": { + "optional": true + } + } + }, + "node_modules/for-in": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/for-in/-/for-in-1.0.2.tgz", + "integrity": "sha512-7EwmXrOjyL+ChxMhmG5lnW9MPt1aIeZEwKhQzoBUdTV0N3zuwWDZYVJatDvZ2OyzPUvdIAZDsCetk3coyMfcnQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/for-own": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/for-own/-/for-own-1.0.0.tgz", + "integrity": "sha512-0OABksIGrxKK8K4kynWkQ7y1zounQxP+CWnyclVwj81KW3vlLlGUx57DKGcP/LH216GzqnstnPocF16Nxs0Ycg==", + "dev": true, + "dependencies": { + "for-in": "^1.0.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/foreground-child": { + "version": "1.5.6", + "resolved": "https://registry.npmjs.org/foreground-child/-/foreground-child-1.5.6.tgz", + "integrity": "sha512-3TOY+4TKV0Ml83PXJQY+JFQaHNV38lzQDIzzXYg1kWdBLenGgoZhAs0CKgzI31vi2pWEpQMq/Yi4bpKwCPkw7g==", + "dev": true, + "dependencies": { + "cross-spawn": "^4", + "signal-exit": "^3.0.0" + } + }, + "node_modules/foreground-child/node_modules/cross-spawn": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-4.0.2.tgz", + "integrity": "sha512-yAXz/pA1tD8Gtg2S98Ekf/sewp3Lcp3YoFKJ4Hkp5h5yLWnKVTDU0kwjKJ8NDCYcfTLfyGkzTikst+jWypT1iA==", + "dev": true, + "dependencies": { + "lru-cache": "^4.0.1", + "which": "^1.2.9" + } + }, + "node_modules/foreground-child/node_modules/which": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/which/-/which-1.3.1.tgz", + "integrity": "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==", + "dev": true, + "dependencies": { + "isexe": "^2.0.0" + }, + "bin": { + "which": "bin/which" + } + }, + "node_modules/form-data": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.0.tgz", + "integrity": "sha512-ETEklSGi5t0QMZuiXoA/Q6vcnxcLQP5vdugSpuAyi6SVGi2clPPp+xgEhuMaHC+zGgn31Kd235W35f7Hykkaww==", + "dependencies": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.8", + "mime-types": "^2.1.12" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/formidable": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/formidable/-/formidable-2.1.1.tgz", + "integrity": "sha512-0EcS9wCFEzLvfiks7omJ+SiYJAiD+TzK4Pcw1UlUoGnhUxDcMKjt0P7x8wEb0u6OHu8Nb98WG3nxtlF5C7bvUQ==", + "dev": true, + "dependencies": { + "dezalgo": "^1.0.4", + "hexoid": "^1.0.0", + "once": "^1.4.0", + "qs": "^6.11.0" + }, + "funding": { + "url": "https://ko-fi.com/tunnckoCore/commissions" + } + }, + "node_modules/formidable/node_modules/qs": { + "version": "6.11.1", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.11.1.tgz", + "integrity": "sha512-0wsrzgTz/kAVIeuxSjnpGC56rzYtr6JT/2BwEvMaPhFIoYa1aGO8LbzuU1R0uUYQkLpWBTOj0l/CLAJB64J6nQ==", + "dev": true, + "dependencies": { + "side-channel": "^1.0.4" + }, + "engines": { + "node": ">=0.6" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/forwarded": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz", + "integrity": "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/fresh": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz", + "integrity": "sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/fs-constants": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz", + "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==", + "dev": true + }, + "node_modules/fs-extra": { + "version": "10.0.0", + "resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-10.0.0.tgz", + "integrity": "sha512-C5owb14u9eJwizKGdchcDUQeFtlSHHthBk8pbX9Vc1PFZrLombudjDnNns88aYslCyF6IY5SUw3Roz6xShcEIQ==", + "dependencies": { + "graceful-fs": "^4.2.0", + "jsonfile": "^6.0.1", + "universalify": "^2.0.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/fs-minipass": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/fs-minipass/-/fs-minipass-2.1.0.tgz", + "integrity": "sha512-V/JgOLFCS+R6Vcq0slCuaeWEdNC3ouDlJMNIsacH2VtALiu9mV4LPrHc5cDl8k5aw6J8jwgWWpiTo5RYhmIzvg==", + "dependencies": { + "minipass": "^3.0.0" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/fs-then-native": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/fs-then-native/-/fs-then-native-2.0.0.tgz", + "integrity": "sha512-X712jAOaWXkemQCAmWeg5rOT2i+KOpWz1Z/txk/cW0qlOu2oQ9H61vc5w3X/iyuUEfq/OyaFJ78/cZAQD1/bgA==", + "dev": true, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/fs.notify": { + "version": "0.0.4", + "resolved": "https://registry.npmjs.org/fs.notify/-/fs.notify-0.0.4.tgz", + "integrity": "sha512-xnulkRf31FQwC8NsU5DEYqMTeM3jZpYsTC2hHQcHlkXTubxQHDVWkau13U/oFmFXieCkai2oKTa1MhckXk2fRQ==", + "dependencies": { + "async": "~0.1.22", + "retry": "~0.6.0" + } + }, + "node_modules/fs.realpath": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz", + "integrity": "sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw==" + }, + "node_modules/fsevents": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", + "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==", + "dev": true, + "hasInstallScript": true, + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/function-bind": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz", + "integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==", + "dev": true + }, + "node_modules/gauge": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/gauge/-/gauge-3.0.2.tgz", + "integrity": "sha512-+5J6MS/5XksCuXq++uFRsnUd7Ovu1XenbeuIuNRJxYWjgQbPuFhT14lAvsWfqfAmnwluf1OwMjz39HjfLPci0Q==", + "optional": true, + "dependencies": { + "aproba": "^1.0.3 || ^2.0.0", + "color-support": "^1.1.2", + "console-control-strings": "^1.0.0", + "has-unicode": "^2.0.1", + "object-assign": "^4.1.1", + "signal-exit": "^3.0.0", + "string-width": "^4.2.3", + "strip-ansi": "^6.0.1", + "wide-align": "^1.1.2" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/gaze": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/gaze/-/gaze-1.1.3.tgz", + "integrity": "sha512-BRdNm8hbWzFzWHERTrejLqwHDfS4GibPoq5wjTPIoJHoBtKGPg3xAFfxmM+9ztbXelxcf2hwQcaz1PtmFeue8g==", + "dev": true, + "dependencies": { + "globule": "^1.0.0" + }, + "engines": { + "node": ">= 4.0.0" + } + }, + "node_modules/get-caller-file": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz", + "integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==", + "dev": true, + "engines": { + "node": "6.* || 8.* || >= 10.*" + } + }, + "node_modules/get-intrinsic": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.2.0.tgz", + "integrity": "sha512-L049y6nFOuom5wGyRc3/gdTLO94dySVKRACj1RmJZBQXlbTMhtNIgkWkUHq+jYmZvKf14EW1EoJnnjbmoHij0Q==", + "dev": true, + "dependencies": { + "function-bind": "^1.1.1", + "has": "^1.0.3", + "has-symbols": "^1.0.3" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/get-stream": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-5.2.0.tgz", + "integrity": "sha512-nBF+F1rAZVCu/p7rjzgA+Yb4lfYXrpl7a6VmJrU8wF9I1CKvP/QwPNZHnOlwbTkY6dvtFIzFMSyQXbLoTQPRpA==", + "dependencies": { + "pump": "^3.0.0" + }, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/getobject": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/getobject/-/getobject-1.0.2.tgz", + "integrity": "sha512-2zblDBaFcb3rB4rF77XVnuINOE2h2k/OnqXAiy0IrTxUfV1iFp3la33oAQVY9pCpWU268WFYVt2t71hlMuLsOg==", + "dev": true, + "engines": { + "node": ">=10" + } + }, + "node_modules/glob": { + "version": "7.1.7", + "resolved": "https://registry.npmjs.org/glob/-/glob-7.1.7.tgz", + "integrity": "sha512-OvD9ENzPLbegENnYP5UUfJIirTg4+XwMWGaQfQTY0JenxNvvIKP3U3/tAQSPIu/lHxXYSZmpXlUHeqAIdKzBLQ==", + "dependencies": { + "fs.realpath": "^1.0.0", + "inflight": "^1.0.4", + "inherits": "2", + "minimatch": "^3.0.4", + "once": "^1.3.0", + "path-is-absolute": "^1.0.0" + }, + "engines": { + "node": "*" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/glob-parent": { + "version": "5.1.2", + "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", + "integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==", + "dev": true, + "dependencies": { + "is-glob": "^4.0.1" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/global-dirs": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/global-dirs/-/global-dirs-3.0.1.tgz", + "integrity": "sha512-NBcGGFbBA9s1VzD41QXDG+3++t9Mn5t1FpLdhESY6oKY4gYTFpX4wO3sqGUa0Srjtbfj3szX0RnemmrVRUdULA==", + "dev": true, + "dependencies": { + "ini": "2.0.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/global-modules": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/global-modules/-/global-modules-1.0.0.tgz", + "integrity": "sha512-sKzpEkf11GpOFuw0Zzjzmt4B4UZwjOcG757PPvrfhxcLFbq0wpsgpOqxpxtxFiCG4DtG93M6XRVbF2oGdev7bg==", + "dev": true, + "dependencies": { + "global-prefix": "^1.0.1", + "is-windows": "^1.0.1", + "resolve-dir": "^1.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/global-prefix": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/global-prefix/-/global-prefix-1.0.2.tgz", + "integrity": "sha512-5lsx1NUDHtSjfg0eHlmYvZKv8/nVqX4ckFbM+FrGcQ+04KWcWFo9P5MxPZYSzUvyzmdTbI7Eix8Q4IbELDqzKg==", + "dev": true, + "dependencies": { + "expand-tilde": "^2.0.2", + "homedir-polyfill": "^1.0.1", + "ini": "^1.3.4", + "is-windows": "^1.0.1", + "which": "^1.2.14" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/global-prefix/node_modules/ini": { + "version": "1.3.8", + "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz", + "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==", + "dev": true + }, + "node_modules/global-prefix/node_modules/which": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/which/-/which-1.3.1.tgz", + "integrity": "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==", + "dev": true, + "dependencies": { + "isexe": "^2.0.0" + }, + "bin": { + "which": "bin/which" + } + }, + "node_modules/globals": { + "version": "11.12.0", + "resolved": "https://registry.npmjs.org/globals/-/globals-11.12.0.tgz", + "integrity": "sha512-WOBp/EEGUiIsJSp7wcv/y6MO+lV9UoncWqxuFfm8eBwzWNgyfBd6Gz+IeKQ9jCmyhoH99g15M3T+QaVHFjizVA==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/globule": { + "version": "1.3.4", + "resolved": "https://registry.npmjs.org/globule/-/globule-1.3.4.tgz", + "integrity": "sha512-OPTIfhMBh7JbBYDpa5b+Q5ptmMWKwcNcFSR/0c6t8V4f3ZAVBEsKNY37QdVqmLRYSMhOUGYrY0QhSoEpzGr/Eg==", + "dev": true, + "dependencies": { + "glob": "~7.1.1", + "lodash": "^4.17.21", + "minimatch": "~3.0.2" + }, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/got": { + "version": "11.8.3", + "resolved": "https://registry.npmjs.org/got/-/got-11.8.3.tgz", + "integrity": "sha512-7gtQ5KiPh1RtGS9/Jbv1ofDpBFuq42gyfEib+ejaRBJuj/3tQFeR5+gw57e4ipaU8c/rCjvX6fkQz2lyDlGAOg==", + "dependencies": { + "@sindresorhus/is": "^4.0.0", + "@szmarczak/http-timer": "^4.0.5", + "@types/cacheable-request": "^6.0.1", + "@types/responselike": "^1.0.0", + "cacheable-lookup": "^5.0.3", + "cacheable-request": "^7.0.2", + "decompress-response": "^6.0.0", + "http2-wrapper": "^1.0.0-beta.5.2", + "lowercase-keys": "^2.0.0", + "p-cancelable": "^2.0.0", + "responselike": "^2.0.0" + }, + "engines": { + "node": ">=10.19.0" + }, + "funding": { + "url": "https://github.com/sindresorhus/got?sponsor=1" + } + }, + "node_modules/graceful-fs": { + "version": "4.2.11", + "resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz", + "integrity": "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==" + }, + "node_modules/growl": { + "version": "1.10.5", + "resolved": "https://registry.npmjs.org/growl/-/growl-1.10.5.tgz", + "integrity": "sha512-qBr4OuELkhPenW6goKVXiv47US3clb3/IbuWF9KNKEijAy9oeHxU9IgzjvJhHkUzhaj7rOUD7+YGWqUjLp5oSA==", + "dev": true, + "engines": { + "node": ">=4.x" + } + }, + "node_modules/grunt": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/grunt/-/grunt-1.5.2.tgz", + "integrity": "sha512-XCtfaIu72OyDqK24MjWiGC9SwlkuhkS1mrULr1xzuJ2XqAFhP3ZAchZGHJeSCY6mkaOXU4F7SbmmCF7xIVoC9w==", + "dev": true, + "dependencies": { + "dateformat": "~3.0.3", + "eventemitter2": "~0.4.13", + "exit": "~0.1.2", + "findup-sync": "~0.3.0", + "glob": "~7.1.6", + "grunt-cli": "~1.4.3", + "grunt-known-options": "~2.0.0", + "grunt-legacy-log": "~3.0.0", + "grunt-legacy-util": "~2.0.1", + "iconv-lite": "~0.4.13", + "js-yaml": "~3.14.0", + "minimatch": "~3.0.4", + "mkdirp": "~1.0.4", + "nopt": "~3.0.6", + "rimraf": "~3.0.2" + }, + "bin": { + "grunt": "bin/grunt" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/grunt-chmod": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/grunt-chmod/-/grunt-chmod-1.1.1.tgz", + "integrity": "sha512-f807W/VOIhhaOW85JyeRd4DgB0RcbsGQV/4IvtcKctOWGvPJns4AqN7xW73PG9+RwDnSGxApS+6Xov5L2LeNXg==", + "dev": true, + "dependencies": { + "shelljs": "^0.5.3" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/grunt-cli": { + "version": "1.4.3", + "resolved": "https://registry.npmjs.org/grunt-cli/-/grunt-cli-1.4.3.tgz", + "integrity": "sha512-9Dtx/AhVeB4LYzsViCjUQkd0Kw0McN2gYpdmGYKtE2a5Yt7v1Q+HYZVWhqXc/kGnxlMtqKDxSwotiGeFmkrCoQ==", + "dev": true, + "dependencies": { + "grunt-known-options": "~2.0.0", + "interpret": "~1.1.0", + "liftup": "~3.0.1", + "nopt": "~4.0.1", + "v8flags": "~3.2.0" + }, + "bin": { + "grunt": "bin/grunt" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/grunt-cli/node_modules/nopt": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/nopt/-/nopt-4.0.3.tgz", + "integrity": "sha512-CvaGwVMztSMJLOeXPrez7fyfObdZqNUK1cPAEzLHrTybIua9pMdmmPR5YwtfNftIOMv3DPUhFaxsZMNTQO20Kg==", + "dev": true, + "dependencies": { + "abbrev": "1", + "osenv": "^0.1.4" + }, + "bin": { + "nopt": "bin/nopt.js" + } + }, + "node_modules/grunt-concurrent": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/grunt-concurrent/-/grunt-concurrent-3.0.0.tgz", + "integrity": "sha512-AgXtjUJESHEGeGX8neL3nmXBTHSj1QC48ABQ3ng2/vjuSBpDD8gKcVHSlXP71pFkIR8TQHf+eomOx6OSYSgfrA==", + "dev": true, + "dependencies": { + "arrify": "^2.0.1", + "async": "^3.1.0", + "indent-string": "^4.0.0", + "pad-stream": "^2.0.0" + }, + "engines": { + "node": ">=8" + }, + "peerDependencies": { + "grunt": ">=1" + } + }, + "node_modules/grunt-concurrent/node_modules/async": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/async/-/async-3.2.4.tgz", + "integrity": "sha512-iAB+JbDEGXhyIUavoDl9WP/Jj106Kz9DEn1DPgYw5ruDn0e3Wgi3sKFm55sASdGBNOQB8F59d9qQ7deqrHA8wQ==", + "dev": true + }, + "node_modules/grunt-contrib-clean": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/grunt-contrib-clean/-/grunt-contrib-clean-2.0.1.tgz", + "integrity": "sha512-uRvnXfhiZt8akb/ZRDHJpQQtkkVkqc/opWO4Po/9ehC2hPxgptB9S6JHDC/Nxswo4CJSM0iFPT/Iym3cEMWzKA==", + "dev": true, + "dependencies": { + "async": "^3.2.3", + "rimraf": "^2.6.2" + }, + "engines": { + "node": ">=12" + }, + "peerDependencies": { + "grunt": ">=0.4.5" + } + }, + "node_modules/grunt-contrib-clean/node_modules/async": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/async/-/async-3.2.4.tgz", + "integrity": "sha512-iAB+JbDEGXhyIUavoDl9WP/Jj106Kz9DEn1DPgYw5ruDn0e3Wgi3sKFm55sASdGBNOQB8F59d9qQ7deqrHA8wQ==", + "dev": true + }, + "node_modules/grunt-contrib-clean/node_modules/rimraf": { + "version": "2.7.1", + "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-2.7.1.tgz", + "integrity": "sha512-uWjbaKIK3T1OSVptzX7Nl6PvQ3qAGtKEtVRjRuazjfL3Bx5eI409VZSqgND+4UNnmzLVdPj9FqFJNPqBZFve4w==", + "dev": true, + "dependencies": { + "glob": "^7.1.3" + }, + "bin": { + "rimraf": "bin.js" + } + }, + "node_modules/grunt-contrib-compress": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/grunt-contrib-compress/-/grunt-contrib-compress-2.0.0.tgz", + "integrity": "sha512-r/dAGx4qG+rmBFF4lb/hTktW2huGMGxkSLf9msh3PPtq0+cdQRQerZJ30UKevX3BLQsohwLzO0p1z/LrH6aKXQ==", + "dev": true, + "dependencies": { + "adm-zip": "^0.5.1", + "archiver": "^5.1.0", + "chalk": "^4.1.0", + "lodash": "^4.17.20", + "pretty-bytes": "^5.4.1", + "stream-buffers": "^3.0.2" + }, + "engines": { + "node": ">=10.16" + } + }, + "node_modules/grunt-contrib-concat": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/grunt-contrib-concat/-/grunt-contrib-concat-1.0.1.tgz", + "integrity": "sha512-QdTmcxe8aim2Z0dFeuSJ+f7fHIeY7PZaTMZxgvosjXwyMhpy2GUR5WHkr12lksHfZVE80v2wUwqF56wyfPUwoQ==", + "dev": true, + "dependencies": { + "chalk": "^1.0.0", + "source-map": "^0.5.3" + }, + "engines": { + "node": ">=0.10.0" + }, + "peerDependencies": { + "grunt": ">=0.4.0" + } + }, + "node_modules/grunt-contrib-concat/node_modules/ansi-regex": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz", + "integrity": "sha512-TIGnTpdo+E3+pCyAluZvtED5p5wCqLdezCyhPZzKPcxvFplEt4i+W7OONCKgeZFT3+y5NZZfOOS/Bdcanm1MYA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-concat/node_modules/ansi-styles": { + "version": "2.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-2.2.1.tgz", + "integrity": "sha512-kmCevFghRiWM7HB5zTPULl4r9bVFSWjz62MhqizDGUrq2NWuNMQyuv4tHHoKJHs69M/MF64lEcHdYIocrdWQYA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-concat/node_modules/chalk": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-1.1.3.tgz", + "integrity": "sha512-U3lRVLMSlsCfjqYPbLyVv11M9CPW4I728d6TCKMAOJueEeB9/8o+eSsMnxPJD+Q+K909sdESg7C+tIkoH6on1A==", + "dev": true, + "dependencies": { + "ansi-styles": "^2.2.1", + "escape-string-regexp": "^1.0.2", + "has-ansi": "^2.0.0", + "strip-ansi": "^3.0.0", + "supports-color": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-concat/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/grunt-contrib-concat/node_modules/strip-ansi": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz", + "integrity": "sha512-VhumSSbBqDTP8p2ZLKj40UjBCV4+v8bUSEpUb4KjRgWk9pbqGF4REFj6KEagidb2f/M6AzC0EmFyDNGaw9OCzg==", + "dev": true, + "dependencies": { + "ansi-regex": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-concat/node_modules/supports-color": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-2.0.0.tgz", + "integrity": "sha512-KKNVtd6pCYgPIKU4cp2733HWYCpplQhddZLBUryaAHou723x+FRzQ5Df824Fj+IyyuiQTRoub4SnIFfIcrp70g==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/grunt-contrib-copy": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/grunt-contrib-copy/-/grunt-contrib-copy-1.0.0.tgz", + "integrity": "sha512-gFRFUB0ZbLcjKb67Magz1yOHGBkyU6uL29hiEW1tdQ9gQt72NuMKIy/kS6dsCbV0cZ0maNCb0s6y+uT1FKU7jA==", + "dev": true, + "dependencies": { + "chalk": "^1.1.1", + "file-sync-cmp": "^0.1.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-copy/node_modules/ansi-regex": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz", + "integrity": "sha512-TIGnTpdo+E3+pCyAluZvtED5p5wCqLdezCyhPZzKPcxvFplEt4i+W7OONCKgeZFT3+y5NZZfOOS/Bdcanm1MYA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-copy/node_modules/ansi-styles": { + "version": "2.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-2.2.1.tgz", + "integrity": "sha512-kmCevFghRiWM7HB5zTPULl4r9bVFSWjz62MhqizDGUrq2NWuNMQyuv4tHHoKJHs69M/MF64lEcHdYIocrdWQYA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-copy/node_modules/chalk": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-1.1.3.tgz", + "integrity": "sha512-U3lRVLMSlsCfjqYPbLyVv11M9CPW4I728d6TCKMAOJueEeB9/8o+eSsMnxPJD+Q+K909sdESg7C+tIkoH6on1A==", + "dev": true, + "dependencies": { + "ansi-styles": "^2.2.1", + "escape-string-regexp": "^1.0.2", + "has-ansi": "^2.0.0", + "strip-ansi": "^3.0.0", + "supports-color": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-copy/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/grunt-contrib-copy/node_modules/strip-ansi": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz", + "integrity": "sha512-VhumSSbBqDTP8p2ZLKj40UjBCV4+v8bUSEpUb4KjRgWk9pbqGF4REFj6KEagidb2f/M6AzC0EmFyDNGaw9OCzg==", + "dev": true, + "dependencies": { + "ansi-regex": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-copy/node_modules/supports-color": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-2.0.0.tgz", + "integrity": "sha512-KKNVtd6pCYgPIKU4cp2733HWYCpplQhddZLBUryaAHou723x+FRzQ5Df824Fj+IyyuiQTRoub4SnIFfIcrp70g==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/grunt-contrib-jshint": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/grunt-contrib-jshint/-/grunt-contrib-jshint-3.1.1.tgz", + "integrity": "sha512-EwMY6L91FqTcMlZTVoDeeq/EZL+7MoFyo1rxIea9sxyv73geVggeE37jcUhNbu5hLbxHE82CGIUqitHuR2/q+g==", + "dev": true, + "dependencies": { + "chalk": "^4.1.0", + "hooker": "^0.2.3", + "jshint": "~2.13.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/grunt-contrib-uglify": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/grunt-contrib-uglify/-/grunt-contrib-uglify-5.0.1.tgz", + "integrity": "sha512-T/aXZ4WIpAtoswZqb6HROKg7uq9QbKwl+lUuOwK4eoFj3tFv9/a/oMyd3/qvetV29Pbf8P1YYda1gDwZppr60A==", + "dev": true, + "dependencies": { + "chalk": "^2.4.1", + "maxmin": "^2.1.0", + "uglify-js": "^3.13.3", + "uri-path": "^1.0.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/grunt-contrib-uglify/node_modules/ansi-styles": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz", + "integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==", + "dev": true, + "dependencies": { + "color-convert": "^1.9.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/grunt-contrib-uglify/node_modules/chalk": { + "version": "2.4.2", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz", + "integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==", + "dev": true, + "dependencies": { + "ansi-styles": "^3.2.1", + "escape-string-regexp": "^1.0.5", + "supports-color": "^5.3.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/grunt-contrib-uglify/node_modules/color-convert": { + "version": "1.9.3", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.3.tgz", + "integrity": "sha512-QfAUtd+vFdAtFQcC8CCyYt1fYWxSqAiK2cSD6zDB8N3cpsEBAvRxp9zOGg6G/SHHJYAT88/az/IuDGALsNVbGg==", + "dev": true, + "dependencies": { + "color-name": "1.1.3" + } + }, + "node_modules/grunt-contrib-uglify/node_modules/color-name": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz", + "integrity": "sha512-72fSenhMw2HZMTVHeCA9KCmpEIbzWiQsjN+BHcBbS9vr1mtt+vJjPdksIBNUmKAW8TFUDPJK5SUU3QhE9NEXDw==", + "dev": true + }, + "node_modules/grunt-contrib-uglify/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/grunt-contrib-uglify/node_modules/has-flag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", + "integrity": "sha512-sKJf1+ceQBr4SMkvQnBDNDtf4TXpVhVGateu0t918bl30FnbE2m4vNLX+VWe/dpjlb+HugGYzW7uQXH98HPEYw==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/grunt-contrib-uglify/node_modules/supports-color": { + "version": "5.5.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz", + "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==", + "dev": true, + "dependencies": { + "has-flag": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/grunt-contrib-watch": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/grunt-contrib-watch/-/grunt-contrib-watch-1.1.0.tgz", + "integrity": "sha512-yGweN+0DW5yM+oo58fRu/XIRrPcn3r4tQx+nL7eMRwjpvk+rQY6R8o94BPK0i2UhTg9FN21hS+m8vR8v9vXfeg==", + "dev": true, + "dependencies": { + "async": "^2.6.0", + "gaze": "^1.1.0", + "lodash": "^4.17.10", + "tiny-lr": "^1.1.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-contrib-watch/node_modules/async": { + "version": "2.6.4", + "resolved": "https://registry.npmjs.org/async/-/async-2.6.4.tgz", + "integrity": "sha512-mzo5dfJYwAn29PeiJ0zvwTo04zj8HDJj0Mn8TD7sno7q12prdbnasKJHhkm2c1LgrhlJ0teaea8860oxi51mGA==", + "dev": true, + "dependencies": { + "lodash": "^4.17.14" + } + }, + "node_modules/grunt-jsdoc": { + "version": "2.4.1", + "resolved": "https://registry.npmjs.org/grunt-jsdoc/-/grunt-jsdoc-2.4.1.tgz", + "integrity": "sha512-S0zxU0wDewRu7z+vijEItOWe/UttxWVmvz0qz2ZVcAYR2GpXjsiski2CAVN0b18t2qeVLdmxZkJaEWCOsKzcAw==", + "dev": true, + "dependencies": { + "cross-spawn": "^7.0.1", + "jsdoc": "^3.6.3" + }, + "bin": { + "grunt-jsdoc": "bin/grunt-jsdoc" + }, + "engines": { + "node": ">= 8.12.0" + } + }, + "node_modules/grunt-jsdoc-to-markdown": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/grunt-jsdoc-to-markdown/-/grunt-jsdoc-to-markdown-6.0.0.tgz", + "integrity": "sha512-vvanKUErp6CHl4MuLQ9vwJewpMu8Fi7z09lr4OwMLr+GBu3nG5lRNZuu5mkWY8qv1aU8WkX97/rJaVs3A1Wx8g==", + "dev": true, + "dependencies": { + "jsdoc-to-markdown": "^7.0.0" + }, + "engines": { + "node": ">=14" + }, + "peerDependencies": { + "grunt": ">=1.3.0" + } + }, + "node_modules/grunt-jsonlint": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/grunt-jsonlint/-/grunt-jsonlint-2.1.3.tgz", + "integrity": "sha512-h04qC969LIyhsJaASeJ/hDnnKnsnSNZKqjYHra6cc/WQzrfMZinQPdr11FhU4T8OzAq6ecMX5QauYiJkbjUV/Q==", + "dev": true, + "dependencies": { + "@prantlf/jsonlint": "10.2.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/grunt-known-options": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/grunt-known-options/-/grunt-known-options-2.0.0.tgz", + "integrity": "sha512-GD7cTz0I4SAede1/+pAbmJRG44zFLPipVtdL9o3vqx9IEyb7b4/Y3s7r6ofI3CchR5GvYJ+8buCSioDv5dQLiA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt-legacy-log": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/grunt-legacy-log/-/grunt-legacy-log-3.0.0.tgz", + "integrity": "sha512-GHZQzZmhyq0u3hr7aHW4qUH0xDzwp2YXldLPZTCjlOeGscAOWWPftZG3XioW8MasGp+OBRIu39LFx14SLjXRcA==", + "dev": true, + "dependencies": { + "colors": "~1.1.2", + "grunt-legacy-log-utils": "~2.1.0", + "hooker": "~0.2.3", + "lodash": "~4.17.19" + }, + "engines": { + "node": ">= 0.10.0" + } + }, + "node_modules/grunt-legacy-log-utils": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/grunt-legacy-log-utils/-/grunt-legacy-log-utils-2.1.0.tgz", + "integrity": "sha512-lwquaPXJtKQk0rUM1IQAop5noEpwFqOXasVoedLeNzaibf/OPWjKYvvdqnEHNmU+0T0CaReAXIbGo747ZD+Aaw==", + "dev": true, + "dependencies": { + "chalk": "~4.1.0", + "lodash": "~4.17.19" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/grunt-legacy-util": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/grunt-legacy-util/-/grunt-legacy-util-2.0.1.tgz", + "integrity": "sha512-2bQiD4fzXqX8rhNdXkAywCadeqiPiay0oQny77wA2F3WF4grPJXCvAcyoWUJV+po/b15glGkxuSiQCK299UC2w==", + "dev": true, + "dependencies": { + "async": "~3.2.0", + "exit": "~0.1.2", + "getobject": "~1.0.0", + "hooker": "~0.2.3", + "lodash": "~4.17.21", + "underscore.string": "~3.3.5", + "which": "~2.0.2" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/grunt-legacy-util/node_modules/async": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/async/-/async-3.2.4.tgz", + "integrity": "sha512-iAB+JbDEGXhyIUavoDl9WP/Jj106Kz9DEn1DPgYw5ruDn0e3Wgi3sKFm55sASdGBNOQB8F59d9qQ7deqrHA8wQ==", + "dev": true + }, + "node_modules/grunt-mkdir": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/grunt-mkdir/-/grunt-mkdir-1.1.0.tgz", + "integrity": "sha512-FRE17OYVveNbVJFX8GPGa5bzH2ZiAdBx3q0Kwk2Dg6l+TzLGaTdufUxiUWUbS2MERFacnmXZwDDOR5ZbYW0o+Q==", + "dev": true, + "engines": { + "node": ">= 0.8.0" + }, + "peerDependencies": { + "grunt": ">=0.4.0" + } + }, + "node_modules/grunt-npm-command": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/grunt-npm-command/-/grunt-npm-command-0.1.2.tgz", + "integrity": "sha512-QsGLL8Pp+tzeIkCqohIbOtVopOhINErRVpxKY+SnvSEE3BXOKKSanlIh9cd1mliajO57sXG2ZC4R8L3v2NSPTQ==", + "dev": true, + "peerDependencies": { + "grunt": ">=0.4.0" + } + }, + "node_modules/grunt-sass": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/grunt-sass/-/grunt-sass-3.1.0.tgz", + "integrity": "sha512-90s27H7FoCDcA8C8+R0GwC+ntYD3lG6S/jqcavWm3bn9RiJTmSfOvfbFa1PXx4NbBWuiGQMLfQTj/JvvqT5w6A==", + "dev": true, + "engines": { + "node": ">=8" + }, + "peerDependencies": { + "grunt": ">=1" + } + }, + "node_modules/grunt-simple-mocha": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/grunt-simple-mocha/-/grunt-simple-mocha-0.4.1.tgz", + "integrity": "sha512-EibTuZVvyLd9v/9An+5sL+XLoArs1QkFSTUcOG/AbBzeCYemZppcO9YSEspWUwU/T/NNtAyzB+x7B6zAmKQqkA==", + "dev": true, + "dependencies": { + "mocha": "*" + }, + "bin": { + "grunt-simple-mocha": "bin/grunt-simple-mocha" + }, + "engines": { + "node": "*" + } + }, + "node_modules/grunt-simple-nyc": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/grunt-simple-nyc/-/grunt-simple-nyc-3.0.1.tgz", + "integrity": "sha512-/YLY+jNI6gBuVO3xu07zwvDN+orTAFS50W00yb/2ncvc2PFO4pR+oU7TyiHhe8a6O3KuQDHsyCE0iE+rqJagQg==", + "dev": true, + "dependencies": { + "lodash": "^4.17.15", + "nyc": "^14.1.0", + "simple-cli": "^5.0.3" + } + }, + "node_modules/grunt/node_modules/iconv-lite": { + "version": "0.4.24", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz", + "integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==", + "dev": true, + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/grunt/node_modules/nopt": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/nopt/-/nopt-3.0.6.tgz", + "integrity": "sha512-4GUt3kSEYmk4ITxzB/b9vaIDfUVWN/Ml1Fwl11IlnIG2iaJ9O6WXZ9SrYM9NLI8OCBieN2Y8SWC2oJV0RQ7qYg==", + "dev": true, + "dependencies": { + "abbrev": "1" + }, + "bin": { + "nopt": "bin/nopt.js" + } + }, + "node_modules/gzip-size": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/gzip-size/-/gzip-size-3.0.0.tgz", + "integrity": "sha512-6s8trQiK+OMzSaCSVXX+iqIcLV9tC+E73jrJrJTyS4h/AJhlxHvzFKqM1YLDJWRGgHX8uLkBeXkA0njNj39L4w==", + "dev": true, + "dependencies": { + "duplexer": "^0.1.1" + }, + "engines": { + "node": ">=0.12.0" + } + }, + "node_modules/handlebars": { + "version": "4.7.7", + "resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.7.7.tgz", + "integrity": "sha512-aAcXm5OAfE/8IXkcZvCepKU3VzW1/39Fb5ZuqMtgI/hT8X2YgoMvBY5dLhq/cpOvw7Lk1nK/UF71aLG/ZnVYRA==", + "dev": true, + "dependencies": { + "minimist": "^1.2.5", + "neo-async": "^2.6.0", + "source-map": "^0.6.1", + "wordwrap": "^1.0.0" + }, + "bin": { + "handlebars": "bin/handlebars" + }, + "engines": { + "node": ">=0.4.7" + }, + "optionalDependencies": { + "uglify-js": "^3.1.4" + } + }, + "node_modules/handlebars/node_modules/source-map": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz", + "integrity": "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/has": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz", + "integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==", + "dev": true, + "dependencies": { + "function-bind": "^1.1.1" + }, + "engines": { + "node": ">= 0.4.0" + } + }, + "node_modules/has-ansi": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/has-ansi/-/has-ansi-2.0.0.tgz", + "integrity": "sha512-C8vBJ8DwUCx19vhm7urhTuUsr4/IyP6l4VzNQDv+ryHQObW3TTTp9yB68WpYgRe2bbaGuZ/se74IqFeVnMnLZg==", + "dev": true, + "dependencies": { + "ansi-regex": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/has-ansi/node_modules/ansi-regex": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz", + "integrity": "sha512-TIGnTpdo+E3+pCyAluZvtED5p5wCqLdezCyhPZzKPcxvFplEt4i+W7OONCKgeZFT3+y5NZZfOOS/Bdcanm1MYA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/has-flag": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz", + "integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/has-symbols": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.0.3.tgz", + "integrity": "sha512-l3LCuF6MgDNwTDKkdYGEihYjt5pRPbEg46rtlmnSPlUbgmB8LOIrKJbYYFBSbnPaJexMKtiPO8hmeRjRz2Td+A==", + "dev": true, + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/has-unicode": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/has-unicode/-/has-unicode-2.0.1.tgz", + "integrity": "sha512-8Rf9Y83NBReMnx0gFzA8JImQACstCYWUplepDa9xprwwtmgEZUF0h/i5xSA625zB/I37EtrswSST6OXxwaaIJQ==", + "optional": true + }, + "node_modules/has-yarn": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/has-yarn/-/has-yarn-2.1.0.tgz", + "integrity": "sha512-UqBRqi4ju7T+TqGNdqAO0PaSVGsDGJUBQvk9eUWNGRY1CFGDzYhLWoM7JQEemnlvVcv/YEmc2wNW8BC24EnUsw==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/hash-sum": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/hash-sum/-/hash-sum-2.0.0.tgz", + "integrity": "sha512-WdZTbAByD+pHfl/g9QSsBIIwy8IT+EsPiKDs0KNX+zSHhdDLFKdZu0BQHljvO+0QI/BasbMSUa8wYNCZTvhslg==" + }, + "node_modules/hasha": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/hasha/-/hasha-3.0.0.tgz", + "integrity": "sha512-w0Kz8lJFBoyaurBiNrIvxPqr/gJ6fOfSkpAPOepN3oECqGJag37xPbOv57izi/KP8auHgNYxn5fXtAb+1LsJ6w==", + "dev": true, + "dependencies": { + "is-stream": "^1.0.1" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/he": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/he/-/he-1.2.0.tgz", + "integrity": "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw==", + "dev": true, + "bin": { + "he": "bin/he" + } + }, + "node_modules/help-me": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/help-me/-/help-me-3.0.0.tgz", + "integrity": "sha512-hx73jClhyk910sidBB7ERlnhMlFsJJIBqSVMFDwPN8o2v9nmp5KgLq1Xz1Bf1fCMMZ6mPrX159iG0VLy/fPMtQ==", + "dependencies": { + "glob": "^7.1.6", + "readable-stream": "^3.6.0" + } + }, + "node_modules/hexoid": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/hexoid/-/hexoid-1.0.0.tgz", + "integrity": "sha512-QFLV0taWQOZtvIRIAdBChesmogZrtuXvVWsFHZTk2SU+anspqZ2vMnoLg7IE1+Uk16N19APic1BuF8bC8c2m5g==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/homedir-polyfill": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/homedir-polyfill/-/homedir-polyfill-1.0.3.tgz", + "integrity": "sha512-eSmmWE5bZTK2Nou4g0AI3zZ9rswp7GRKoKXS1BLUkvPviOqs4YTN1djQIqrXy9k5gEtdLPy86JjRwsNM9tnDcA==", + "dev": true, + "dependencies": { + "parse-passwd": "^1.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/hooker": { + "version": "0.2.3", + "resolved": "https://registry.npmjs.org/hooker/-/hooker-0.2.3.tgz", + "integrity": "sha512-t+UerCsQviSymAInD01Pw+Dn/usmz1sRO+3Zk1+lx8eg+WKpD2ulcwWqHHL0+aseRBr+3+vIhiG1K1JTwaIcTA==", + "dev": true, + "engines": { + "node": "*" + } + }, + "node_modules/hosted-git-info": { + "version": "2.8.9", + "resolved": "https://registry.npmjs.org/hosted-git-info/-/hosted-git-info-2.8.9.tgz", + "integrity": "sha512-mxIDAb9Lsm6DoOJ7xH+5+X4y1LU/4Hi50L9C5sIswK3JzULS4bwk1FvjdBgvYR4bzT4tuUQiC15FE2f5HbLvYw==", + "dev": true + }, + "node_modules/hpagent": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/hpagent/-/hpagent-0.1.2.tgz", + "integrity": "sha512-ePqFXHtSQWAFXYmj+JtOTHr84iNrII4/QRlAAPPE+zqnKy4xJo7Ie1Y4kC7AdB+LxLxSTTzBMASsEcy0q8YyvQ==" + }, + "node_modules/html-escaper": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-2.0.2.tgz", + "integrity": "sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg==", + "dev": true + }, + "node_modules/htmlparser2": { + "version": "6.1.0", + "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-6.1.0.tgz", + "integrity": "sha512-gyyPk6rgonLFEDGoeRgQNaEUvdJ4ktTmmUh/h2t7s+M8oPpIPxgNACWa+6ESR57kXstwqPiCut0V8NRpcwgU7A==", + "funding": [ + "https://github.com/fb55/htmlparser2?sponsor=1", + { + "type": "github", + "url": "https://github.com/sponsors/fb55" + } + ], + "dependencies": { + "domelementtype": "^2.0.1", + "domhandler": "^4.0.0", + "domutils": "^2.5.2", + "entities": "^2.0.0" + } + }, + "node_modules/http-cache-semantics": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/http-cache-semantics/-/http-cache-semantics-4.1.1.tgz", + "integrity": "sha512-er295DKPVsV82j5kw1Gjt+ADA/XYHsajl82cGNQG2eyoPkvgUhX+nDIyelzhIWbbsXP39EHcI6l5tYs2FYqYXQ==" + }, + "node_modules/http-errors": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.8.1.tgz", + "integrity": "sha512-Kpk9Sm7NmI+RHhnj6OIWDI1d6fIoFAtFt9RLaTMRlg/8w49juAStsrBgp0Dp4OdxdVbRIeKhtCUvoi/RuAhO4g==", + "dependencies": { + "depd": "~1.1.2", + "inherits": "2.0.4", + "setprototypeof": "1.2.0", + "statuses": ">= 1.5.0 < 2", + "toidentifier": "1.0.1" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/http-parser-js": { + "version": "0.5.8", + "resolved": "https://registry.npmjs.org/http-parser-js/-/http-parser-js-0.5.8.tgz", + "integrity": "sha512-SGeBX54F94Wgu5RH3X5jsDtf4eHyRogWX1XGT3b4HuW3tQPM4AaBzoUji/4AAJNXCEOWZ5O0DgZmJw1947gD5Q==", + "dev": true + }, + "node_modules/http2-wrapper": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/http2-wrapper/-/http2-wrapper-1.0.3.tgz", + "integrity": "sha512-V+23sDMr12Wnz7iTcDeJr3O6AIxlnvT/bmaAAAP/Xda35C90p9599p0F1eHR/N1KILWSoWVAiOMFjBBXaXSMxg==", + "dependencies": { + "quick-lru": "^5.1.1", + "resolve-alpn": "^1.0.0" + }, + "engines": { + "node": ">=10.19.0" + } + }, + "node_modules/https-proxy-agent": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.0.tgz", + "integrity": "sha512-EkYm5BcKUGiduxzSt3Eppko+PiNWNEpa4ySk9vTC6wDsQJW9rHSa+UhGNJoRYp7bz6Ht1eaRIa6QaJqO5rCFbA==", + "dependencies": { + "agent-base": "6", + "debug": "4" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/https-proxy-agent/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/https-proxy-agent/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==" + }, + "node_modules/i18next": { + "version": "21.6.11", + "resolved": "https://registry.npmjs.org/i18next/-/i18next-21.6.11.tgz", + "integrity": "sha512-tJ2+o0lVO+fhi8bPkCpBAeY1SgkqmQm5NzgPWCQssBrywJw98/o+Kombhty5nxQOpHtvMmsxcOopczUiH6bJxQ==", + "funding": [ + { + "type": "individual", + "url": "https://locize.com" + }, + { + "type": "individual", + "url": "https://locize.com/i18next.html" + }, + { + "type": "individual", + "url": "https://www.i18next.com/how-to/faq#i18next-is-awesome.-how-can-i-support-the-project" + } + ], + "dependencies": { + "@babel/runtime": "^7.12.0" + } + }, + "node_modules/i18next-http-backend": { + "version": "1.3.2", + "resolved": "https://registry.npmjs.org/i18next-http-backend/-/i18next-http-backend-1.3.2.tgz", + "integrity": "sha512-SfcoUmsSWnc2LYsDsCq5TCg18cxJXvXymX9N37V+qqMKQY8Gf0rWkjOnRd20sMK633Dq4NF9tvqPbOiFJ49Kbw==", + "dev": true, + "dependencies": { + "cross-fetch": "3.1.5" + } + }, + "node_modules/iconv-lite": { + "version": "0.6.3", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.6.3.tgz", + "integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==", + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/ieee754": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz", + "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ] + }, + "node_modules/ignore-by-default": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/ignore-by-default/-/ignore-by-default-1.0.1.tgz", + "integrity": "sha512-Ius2VYcGNk7T90CppJqcIkS5ooHUZyIQK+ClZfMfMNFEF9VSE73Fq+906u/CWu92x4gzZMWOwfFYckPObzdEbA==", + "dev": true + }, + "node_modules/immutable": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/immutable/-/immutable-4.3.0.tgz", + "integrity": "sha512-0AOCmOip+xgJwEVTQj1EfiDDOkPmuyllDuTuEX+DDXUgapLAsBIfkg3sxCYyCEA8mQqZrrxPUGjcOQ2JS3WLkg==", + "dev": true + }, + "node_modules/import-lazy": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/import-lazy/-/import-lazy-2.1.0.tgz", + "integrity": "sha512-m7ZEHgtw69qOGw+jwxXkHlrlIPdTGkyh66zXZ1ajZbxkDBNjSY/LGbmjc7h0s2ELsUDTAhFr55TrPSSqJGPG0A==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/imurmurhash": { + "version": "0.1.4", + "resolved": "https://registry.npmjs.org/imurmurhash/-/imurmurhash-0.1.4.tgz", + "integrity": "sha512-JmXMZ6wuvDmLiHEml9ykzqO6lwFbof0GG4IkcGaENdCRDDmMVnny7s5HsIgHCbaq0w2MyPhDqkhTUgS2LU2PHA==", + "dev": true, + "engines": { + "node": ">=0.8.19" + } + }, + "node_modules/indent-string": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/indent-string/-/indent-string-4.0.0.tgz", + "integrity": "sha512-EdDDZu4A2OyIK7Lr/2zG+w5jmbuk1DVBnEwREQvBzspBJkCEbRa8GxU1lghYcaGJCnRWibjDXlq779X1/y5xwg==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/inflight": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz", + "integrity": "sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA==", + "dependencies": { + "once": "^1.3.0", + "wrappy": "1" + } + }, + "node_modules/inherits": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz", + "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==" + }, + "node_modules/ini": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/ini/-/ini-2.0.0.tgz", + "integrity": "sha512-7PnF4oN3CvZF23ADhA5wRaYEQpJ8qygSkbtTXWBeXWXmEVRXK+1ITciHWwHhsjv1TmW0MgacIv6hEi5pX5NQdA==", + "dev": true, + "engines": { + "node": ">=10" + } + }, + "node_modules/interpret": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/interpret/-/interpret-1.1.0.tgz", + "integrity": "sha512-CLM8SNMDu7C5psFCn6Wg/tgpj/bKAg7hc2gWqcuR9OD5Ft9PhBpIu8PLicPeis+xDd6YX2ncI8MCA64I9tftIA==", + "dev": true + }, + "node_modules/ipaddr.js": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz", + "integrity": "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==", + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/is-absolute": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-absolute/-/is-absolute-1.0.0.tgz", + "integrity": "sha512-dOWoqflvcydARa360Gvv18DZ/gRuHKi2NU/wU5X1ZFzdYfH29nkiNZsF3mp4OJ3H4yo9Mx8A/uAGNzpzPN3yBA==", + "dev": true, + "dependencies": { + "is-relative": "^1.0.0", + "is-windows": "^1.0.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-arrayish": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/is-arrayish/-/is-arrayish-0.2.1.tgz", + "integrity": "sha512-zz06S8t0ozoDXMG+ube26zeCTNXcKIPJZJi8hBrF4idCLms4CG9QtK7qBl1boi5ODzFpjswb5JPmHCbMpjaYzg==", + "dev": true + }, + "node_modules/is-binary-path": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/is-binary-path/-/is-binary-path-2.1.0.tgz", + "integrity": "sha512-ZMERYes6pDydyuGidse7OsHxtbI7WVeUEozgR/g7rd0xUimYNlvZRE/K2MgZTjWy725IfelLeVcEM97mmtRGXw==", + "dev": true, + "dependencies": { + "binary-extensions": "^2.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/is-ci": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/is-ci/-/is-ci-2.0.0.tgz", + "integrity": "sha512-YfJT7rkpQB0updsdHLGWrvhBJfcfzNNawYDNIyQXJz0IViGf75O8EBPKSdvw2rF+LGCsX4FZ8tcr3b19LcZq4w==", + "dev": true, + "dependencies": { + "ci-info": "^2.0.0" + }, + "bin": { + "is-ci": "bin.js" + } + }, + "node_modules/is-core-module": { + "version": "2.11.0", + "resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.11.0.tgz", + "integrity": "sha512-RRjxlvLDkD1YJwDbroBHMb+cukurkDWNyHx7D3oNB5x9rb5ogcksMC5wHCadcXoo67gVr/+3GFySh3134zi6rw==", + "dev": true, + "dependencies": { + "has": "^1.0.3" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/is-extglob": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz", + "integrity": "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-fullwidth-code-point": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz", + "integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==", + "devOptional": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/is-glob": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/is-glob/-/is-glob-4.0.3.tgz", + "integrity": "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg==", + "dev": true, + "dependencies": { + "is-extglob": "^2.1.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-installed-globally": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/is-installed-globally/-/is-installed-globally-0.4.0.tgz", + "integrity": "sha512-iwGqO3J21aaSkC7jWnHP/difazwS7SFeIqxv6wEtLU8Y5KlzFTjyqcSIT0d8s4+dDhKytsk9PJZ2BkS5eZwQRQ==", + "dev": true, + "dependencies": { + "global-dirs": "^3.0.0", + "is-path-inside": "^3.0.2" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/is-npm": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/is-npm/-/is-npm-5.0.0.tgz", + "integrity": "sha512-WW/rQLOazUq+ST/bCAVBp/2oMERWLsR7OrKyt052dNDk4DHcDE0/7QSXITlmi+VBcV13DfIbysG3tZJm5RfdBA==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/is-number": { + "version": "7.0.0", + "resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz", + "integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==", + "dev": true, + "engines": { + "node": ">=0.12.0" + } + }, + "node_modules/is-obj": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/is-obj/-/is-obj-2.0.0.tgz", + "integrity": "sha512-drqDG3cbczxxEJRoOXcOjtdp1J/lyp1mNn0xaznRs8+muBhgQcrnbspox5X5fOw0HnMnbfDzvnEMEtqDEJEo8w==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/is-path-inside": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/is-path-inside/-/is-path-inside-3.0.3.tgz", + "integrity": "sha512-Fd4gABb+ycGAmKou8eMftCupSir5lRxqf4aD/vd0cD2qc4HL07OjCeuHMr8Ro4CoMaeCKDB0/ECBOVWjTwUvPQ==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/is-plain-obj": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-2.1.0.tgz", + "integrity": "sha512-YWnfyRwxL/+SsrWYfOpUtz5b3YD+nyfkHvjbcanzk8zgyO4ASD67uVMRt8k5bM4lLMDnXfriRhOpemw+NfT1eA==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/is-plain-object": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/is-plain-object/-/is-plain-object-2.0.4.tgz", + "integrity": "sha512-h5PpgXkWitc38BBMYawTYMWJHFZJVnBquFE57xFpjB8pJFiF6gZ+bU+WyI/yqXiFR5mdLsgYNaPe8uao6Uv9Og==", + "dev": true, + "dependencies": { + "isobject": "^3.0.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-relative": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-relative/-/is-relative-1.0.0.tgz", + "integrity": "sha512-Kw/ReK0iqwKeu0MITLFuj0jbPAmEiOsIwyIXvvbfa6QfmN9pkD1M+8pdk7Rl/dTKbH34/XBFMbgD4iMJhLQbGA==", + "dev": true, + "dependencies": { + "is-unc-path": "^1.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-stream": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/is-stream/-/is-stream-1.1.0.tgz", + "integrity": "sha512-uQPm8kcs47jx38atAcWTVxyltQYoPT68y9aWYdV6yWXSyW8mzSat0TL6CiWdZeCdF3KrAvpVtnHbTv4RN+rqdQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-typedarray": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-typedarray/-/is-typedarray-1.0.0.tgz", + "integrity": "sha512-cyA56iCMHAh5CdzjJIa4aohJyeO1YbwLi3Jc35MmRU6poroFjIGZzUzupGiRPOjgHg9TLu43xbpwXk523fMxKA==", + "dev": true + }, + "node_modules/is-unc-path": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-unc-path/-/is-unc-path-1.0.0.tgz", + "integrity": "sha512-mrGpVd0fs7WWLfVsStvgF6iEJnbjDFZh9/emhRDcGWTduTfNHd9CHeUwH3gYIjdbwo4On6hunkztwOaAw0yllQ==", + "dev": true, + "dependencies": { + "unc-path-regex": "^0.1.2" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-unicode-supported": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/is-unicode-supported/-/is-unicode-supported-0.1.0.tgz", + "integrity": "sha512-knxG2q4UC3u8stRGyAVJCOdxFmv5DZiRcdlIaAQXAbSfJya+OhopNotLQrstBhququ4ZpuKbDc/8S6mgXgPFPw==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/is-utf8": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/is-utf8/-/is-utf8-0.2.1.tgz", + "integrity": "sha512-rMYPYvCzsXywIsldgLaSoPlw5PfoB/ssr7hY4pLfcodrA5M/eArza1a9VmTiNIBNMjOGr1Ow9mTyU2o69U6U9Q==" + }, + "node_modules/is-windows": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/is-windows/-/is-windows-1.0.2.tgz", + "integrity": "sha512-eXK1UInq2bPmjyX6e3VHIzMLobc4J94i4AWn+Hpq3OU5KkrRC96OAcR3PRJ/pGu6m8TRnBHP9dkXQVsT/COVIA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-yarn-global": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/is-yarn-global/-/is-yarn-global-0.3.0.tgz", + "integrity": "sha512-VjSeb/lHmkoyd8ryPVIKvOCn4D1koMqY+vqyjjUfc3xyKtP4dYOxM44sZrnqQSzSds3xyOrUTLTC9LVCVgLngw==", + "dev": true + }, + "node_modules/isarray": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz", + "integrity": "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ==" + }, + "node_modules/isexe": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz", + "integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==", + "dev": true + }, + "node_modules/isobject": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/isobject/-/isobject-3.0.1.tgz", + "integrity": "sha512-WhB9zCku7EGTj/HQQRz5aUQEUeoQZH2bWcltRErOpymJ4boYE6wL9Tbr23krRPSZ+C5zqNSrSw+Cc7sZZ4b7vg==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/istanbul-lib-coverage": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/istanbul-lib-coverage/-/istanbul-lib-coverage-2.0.5.tgz", + "integrity": "sha512-8aXznuEPCJvGnMSRft4udDRDtb1V3pkQkMMI5LI+6HuQz5oQ4J2UFn1H82raA3qJtyOLkkwVqICBQkjnGtn5mA==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-hook": { + "version": "2.0.7", + "resolved": "https://registry.npmjs.org/istanbul-lib-hook/-/istanbul-lib-hook-2.0.7.tgz", + "integrity": "sha512-vrRztU9VRRFDyC+aklfLoeXyNdTfga2EI3udDGn4cZ6fpSXpHLV9X6CHvfoMCPtggg8zvDDmC4b9xfu0z6/llA==", + "dev": true, + "dependencies": { + "append-transform": "^1.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-instrument": { + "version": "3.3.0", + "resolved": "https://registry.npmjs.org/istanbul-lib-instrument/-/istanbul-lib-instrument-3.3.0.tgz", + "integrity": "sha512-5nnIN4vo5xQZHdXno/YDXJ0G+I3dAm4XgzfSVTPLQpj/zAV2dV6Juy0yaf10/zrJOJeHoN3fraFe+XRq2bFVZA==", + "dev": true, + "dependencies": { + "@babel/generator": "^7.4.0", + "@babel/parser": "^7.4.3", + "@babel/template": "^7.4.0", + "@babel/traverse": "^7.4.3", + "@babel/types": "^7.4.0", + "istanbul-lib-coverage": "^2.0.5", + "semver": "^6.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-instrument/node_modules/semver": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/semver/-/semver-6.3.0.tgz", + "integrity": "sha512-b39TBaTSfV6yBrapU89p5fKekE2m/NwnDocOVruQFS1/veMgdzuPcnOM34M6CwxW8jH/lxEa5rBoDeUwu5HHTw==", + "dev": true, + "bin": { + "semver": "bin/semver.js" + } + }, + "node_modules/istanbul-lib-report": { + "version": "2.0.8", + "resolved": "https://registry.npmjs.org/istanbul-lib-report/-/istanbul-lib-report-2.0.8.tgz", + "integrity": "sha512-fHBeG573EIihhAblwgxrSenp0Dby6tJMFR/HvlerBsrCTD5bkUuoNtn3gVh29ZCS824cGGBPn7Sg7cNk+2xUsQ==", + "dev": true, + "dependencies": { + "istanbul-lib-coverage": "^2.0.5", + "make-dir": "^2.1.0", + "supports-color": "^6.1.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-report/node_modules/has-flag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", + "integrity": "sha512-sKJf1+ceQBr4SMkvQnBDNDtf4TXpVhVGateu0t918bl30FnbE2m4vNLX+VWe/dpjlb+HugGYzW7uQXH98HPEYw==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/istanbul-lib-report/node_modules/make-dir": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/make-dir/-/make-dir-2.1.0.tgz", + "integrity": "sha512-LS9X+dc8KLxXCb8dni79fLIIUA5VyZoyjSMCwTluaXA0o27cCK0bhXkpgw+sTXVpPy/lSO57ilRixqk0vDmtRA==", + "dev": true, + "dependencies": { + "pify": "^4.0.1", + "semver": "^5.6.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-report/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/istanbul-lib-report/node_modules/supports-color": { + "version": "6.1.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-6.1.0.tgz", + "integrity": "sha512-qe1jfm1Mg7Nq/NSh6XE24gPXROEVsWHxC1LIx//XNlD9iw7YZQGjZNjYN7xGaEG6iKdA8EtNFW6R0gjnVXp+wQ==", + "dev": true, + "dependencies": { + "has-flag": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-source-maps": { + "version": "3.0.6", + "resolved": "https://registry.npmjs.org/istanbul-lib-source-maps/-/istanbul-lib-source-maps-3.0.6.tgz", + "integrity": "sha512-R47KzMtDJH6X4/YW9XTx+jrLnZnscW4VpNN+1PViSYTejLVPWv7oov+Duf8YQSPyVRUvueQqz1TcsC6mooZTXw==", + "dev": true, + "dependencies": { + "debug": "^4.1.1", + "istanbul-lib-coverage": "^2.0.5", + "make-dir": "^2.1.0", + "rimraf": "^2.6.3", + "source-map": "^0.6.1" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-source-maps/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dev": true, + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/istanbul-lib-source-maps/node_modules/make-dir": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/make-dir/-/make-dir-2.1.0.tgz", + "integrity": "sha512-LS9X+dc8KLxXCb8dni79fLIIUA5VyZoyjSMCwTluaXA0o27cCK0bhXkpgw+sTXVpPy/lSO57ilRixqk0vDmtRA==", + "dev": true, + "dependencies": { + "pify": "^4.0.1", + "semver": "^5.6.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/istanbul-lib-source-maps/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==", + "dev": true + }, + "node_modules/istanbul-lib-source-maps/node_modules/rimraf": { + "version": "2.7.1", + "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-2.7.1.tgz", + "integrity": "sha512-uWjbaKIK3T1OSVptzX7Nl6PvQ3qAGtKEtVRjRuazjfL3Bx5eI409VZSqgND+4UNnmzLVdPj9FqFJNPqBZFve4w==", + "dev": true, + "dependencies": { + "glob": "^7.1.3" + }, + "bin": { + "rimraf": "bin.js" + } + }, + "node_modules/istanbul-lib-source-maps/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/istanbul-lib-source-maps/node_modules/source-map": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz", + "integrity": "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/istanbul-reports": { + "version": "2.2.7", + "resolved": "https://registry.npmjs.org/istanbul-reports/-/istanbul-reports-2.2.7.tgz", + "integrity": "sha512-uu1F/L1o5Y6LzPVSVZXNOoD/KXpJue9aeLRd0sM9uMXfZvzomB0WxVamWb5ue8kA2vVWEmW7EG+A5n3f1kqHKg==", + "dev": true, + "dependencies": { + "html-escaper": "^2.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/jquery-i18next": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/jquery-i18next/-/jquery-i18next-1.2.1.tgz", + "integrity": "sha512-UNcw3rgxoKjGEg4w23FEn2h3OlPJU7rPzsgDuXDBZktIzeiVbJohs9Cv9hj8oP8KNfBRKOoErL/OVxg2FaAR4g==", + "dev": true + }, + "node_modules/js-sdsl": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/js-sdsl/-/js-sdsl-4.3.0.tgz", + "integrity": "sha512-mifzlm2+5nZ+lEcLJMoBK0/IH/bDg8XnJfd/Wq6IP+xoCjLZsTOnV2QpxlVbX9bMnkl5PdEjNtBJ9Cj1NjifhQ==", + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/js-sdsl" + } + }, + "node_modules/js-tokens": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz", + "integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==", + "dev": true + }, + "node_modules/js-yaml": { + "version": "3.14.1", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.14.1.tgz", + "integrity": "sha512-okMH7OXXJ7YrN9Ok3/SXrnu4iX9yOk+25nqX4imS2npuvTYDmo/QEZoqwZkYaIDk3jVvBOTOIEgEhaLOynBS9g==", + "dependencies": { + "argparse": "^1.0.7", + "esprima": "^4.0.0" + }, + "bin": { + "js-yaml": "bin/js-yaml.js" + } + }, + "node_modules/js2xmlparser": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/js2xmlparser/-/js2xmlparser-4.0.2.tgz", + "integrity": "sha512-6n4D8gLlLf1n5mNLQPRfViYzu9RATblzPEtm1SthMX1Pjao0r9YI9nw7ZIfRxQMERS87mcswrg+r/OYrPRX6jA==", + "dev": true, + "dependencies": { + "xmlcreate": "^2.0.4" + } + }, + "node_modules/jsdoc": { + "version": "3.6.11", + "resolved": "https://registry.npmjs.org/jsdoc/-/jsdoc-3.6.11.tgz", + "integrity": "sha512-8UCU0TYeIYD9KeLzEcAu2q8N/mx9O3phAGl32nmHlE0LpaJL71mMkP4d+QE5zWfNt50qheHtOZ0qoxVrsX5TUg==", + "dev": true, + "dependencies": { + "@babel/parser": "^7.9.4", + "@types/markdown-it": "^12.2.3", + "bluebird": "^3.7.2", + "catharsis": "^0.9.0", + "escape-string-regexp": "^2.0.0", + "js2xmlparser": "^4.0.2", + "klaw": "^3.0.0", + "markdown-it": "^12.3.2", + "markdown-it-anchor": "^8.4.1", + "marked": "^4.0.10", + "mkdirp": "^1.0.4", + "requizzle": "^0.2.3", + "strip-json-comments": "^3.1.0", + "taffydb": "2.6.2", + "underscore": "~1.13.2" + }, + "bin": { + "jsdoc": "jsdoc.js" + }, + "engines": { + "node": ">=12.0.0" + } + }, + "node_modules/jsdoc-api": { + "version": "7.2.0", + "resolved": "https://registry.npmjs.org/jsdoc-api/-/jsdoc-api-7.2.0.tgz", + "integrity": "sha512-93YDnlm/OYTlLOFeNs4qAv0RBCJ0kGj67xQaWy8wrbk97Rw1EySitoOTHsTHXPEs3uyx2IStPKGrbE7LTnZXbA==", + "dev": true, + "dependencies": { + "array-back": "^6.2.2", + "cache-point": "^2.0.0", + "collect-all": "^1.0.4", + "file-set": "^4.0.2", + "fs-then-native": "^2.0.0", + "jsdoc": "^4.0.0", + "object-to-spawn-args": "^2.0.1", + "temp-path": "^1.0.0", + "walk-back": "^5.1.0" + }, + "engines": { + "node": ">=12.17" + } + }, + "node_modules/jsdoc-api/node_modules/jsdoc": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/jsdoc/-/jsdoc-4.0.2.tgz", + "integrity": "sha512-e8cIg2z62InH7azBBi3EsSEqrKx+nUtAS5bBcYTSpZFA+vhNPyhv8PTFZ0WsjOPDj04/dOLlm08EDcQJDqaGQg==", + "dev": true, + "dependencies": { + "@babel/parser": "^7.20.15", + "@jsdoc/salty": "^0.2.1", + "@types/markdown-it": "^12.2.3", + "bluebird": "^3.7.2", + "catharsis": "^0.9.0", + "escape-string-regexp": "^2.0.0", + "js2xmlparser": "^4.0.2", + "klaw": "^3.0.0", + "markdown-it": "^12.3.2", + "markdown-it-anchor": "^8.4.1", + "marked": "^4.0.10", + "mkdirp": "^1.0.4", + "requizzle": "^0.2.3", + "strip-json-comments": "^3.1.0", + "underscore": "~1.13.2" + }, + "bin": { + "jsdoc": "jsdoc.js" + }, + "engines": { + "node": ">=12.0.0" + } + }, + "node_modules/jsdoc-nr-template": { + "version": "1.0.0", + "resolved": "git+ssh://git@github.com/node-red/jsdoc-nr-template.git#3c7c8f96d585c7c5918a2e63519310e1297e162d", + "dev": true + }, + "node_modules/jsdoc-parse": { + "version": "6.2.0", + "resolved": "https://registry.npmjs.org/jsdoc-parse/-/jsdoc-parse-6.2.0.tgz", + "integrity": "sha512-Afu1fQBEb7QHt6QWX/6eUWvYHJofB90Fjx7FuJYF7mnG9z5BkAIpms1wsnvYLytfmqpEENHs/fax9p8gvMj7dw==", + "dev": true, + "dependencies": { + "array-back": "^6.2.2", + "lodash.omit": "^4.5.0", + "lodash.pick": "^4.4.0", + "reduce-extract": "^1.0.0", + "sort-array": "^4.1.5", + "test-value": "^3.0.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/jsdoc-to-markdown": { + "version": "7.1.1", + "resolved": "https://registry.npmjs.org/jsdoc-to-markdown/-/jsdoc-to-markdown-7.1.1.tgz", + "integrity": "sha512-CI86d63xAVNO+ENumWwmJ034lYe5iGU5GwjtTA11EuphP9tpnoi4hrKgR/J8uME0D+o4KUpVfwX1fjZhc8dEtg==", + "dev": true, + "dependencies": { + "array-back": "^6.2.2", + "command-line-tool": "^0.8.0", + "config-master": "^3.1.0", + "dmd": "^6.1.0", + "jsdoc-api": "^7.1.1", + "jsdoc-parse": "^6.1.0", + "walk-back": "^5.1.0" + }, + "bin": { + "jsdoc2md": "bin/cli.js" + }, + "engines": { + "node": ">=12.17" + } + }, + "node_modules/jsesc": { + "version": "2.5.2", + "resolved": "https://registry.npmjs.org/jsesc/-/jsesc-2.5.2.tgz", + "integrity": "sha512-OYu7XEzjkCQ3C5Ps3QIZsQfNpqoJyZZA99wd9aWd05NCtC5pWOkShK2mkL6HXQR6/Cy2lbNdPlZBpuQHXE63gA==", + "dev": true, + "bin": { + "jsesc": "bin/jsesc" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/jshint": { + "version": "2.13.6", + "resolved": "https://registry.npmjs.org/jshint/-/jshint-2.13.6.tgz", + "integrity": "sha512-IVdB4G0NTTeQZrBoM8C5JFVLjV2KtZ9APgybDA1MK73xb09qFs0jCXyQLnCOp1cSZZZbvhq/6mfXHUTaDkffuQ==", + "dev": true, + "dependencies": { + "cli": "~1.0.0", + "console-browserify": "1.1.x", + "exit": "0.1.x", + "htmlparser2": "3.8.x", + "lodash": "~4.17.21", + "minimatch": "~3.0.2", + "strip-json-comments": "1.0.x" + }, + "bin": { + "jshint": "bin/jshint" + } + }, + "node_modules/jshint/node_modules/dom-serializer": { + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-0.2.2.tgz", + "integrity": "sha512-2/xPb3ORsQ42nHYiSunXkDjPLBaEj/xTwUO4B7XCZQTRk7EBtTOPaygh10YAAh2OI1Qrp6NWfpAhzswj0ydt9g==", + "dev": true, + "dependencies": { + "domelementtype": "^2.0.1", + "entities": "^2.0.0" + } + }, + "node_modules/jshint/node_modules/dom-serializer/node_modules/domelementtype": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz", + "integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fb55" + } + ] + }, + "node_modules/jshint/node_modules/dom-serializer/node_modules/entities": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/entities/-/entities-2.2.0.tgz", + "integrity": "sha512-p92if5Nz619I0w+akJrLZH0MX0Pb5DX39XOwQTtXSdQQOaYH03S1uIQp4mhOZtAXrxq4ViO67YTiLBo2638o9A==", + "dev": true, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, + "node_modules/jshint/node_modules/domelementtype": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-1.3.1.tgz", + "integrity": "sha512-BSKB+TSpMpFI/HOxCNr1O8aMOTZ8hT3pM3GQ0w/mWRmkhEDSFJkkyzz4XQsBV44BChwGkrDfMyjVD0eA2aFV3w==", + "dev": true + }, + "node_modules/jshint/node_modules/domhandler": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-2.3.0.tgz", + "integrity": "sha512-q9bUwjfp7Eif8jWxxxPSykdRZAb6GkguBGSgvvCrhI9wB71W2K/Kvv4E61CF/mcCfnVJDeDWx/Vb/uAqbDj6UQ==", + "dev": true, + "dependencies": { + "domelementtype": "1" + } + }, + "node_modules/jshint/node_modules/domutils": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/domutils/-/domutils-1.5.1.tgz", + "integrity": "sha512-gSu5Oi/I+3wDENBsOWBiRK1eoGxcywYSqg3rR960/+EfY0CF4EX1VPkgHOZ3WiS/Jg2DtliF6BhWcHlfpYUcGw==", + "dev": true, + "dependencies": { + "dom-serializer": "0", + "domelementtype": "1" + } + }, + "node_modules/jshint/node_modules/entities": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/entities/-/entities-1.0.0.tgz", + "integrity": "sha512-LbLqfXgJMmy81t+7c14mnulFHJ170cM6E+0vMXR9k/ZiZwgX8i5pNgjTCX3SO4VeUsFLV+8InixoretwU+MjBQ==", + "dev": true + }, + "node_modules/jshint/node_modules/htmlparser2": { + "version": "3.8.3", + "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-3.8.3.tgz", + "integrity": "sha512-hBxEg3CYXe+rPIua8ETe7tmG3XDn9B0edOE/e9wH2nLczxzgdu0m0aNHY+5wFZiviLWLdANPJTssa92dMcXQ5Q==", + "dev": true, + "dependencies": { + "domelementtype": "1", + "domhandler": "2.3", + "domutils": "1.5", + "entities": "1.0", + "readable-stream": "1.1" + } + }, + "node_modules/jshint/node_modules/isarray": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", + "integrity": "sha512-D2S+3GLxWH+uhrNEcoh/fnmYeP8E8/zHl644d/jdA0g2uyXvy3sb0qxotE+ne0LtccHknQzWwZEzhak7oJ0COQ==", + "dev": true + }, + "node_modules/jshint/node_modules/readable-stream": { + "version": "1.1.14", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz", + "integrity": "sha512-+MeVjFf4L44XUkhM1eYbD8fyEsxcV81pqMSR5gblfcLCHfZvbrqy4/qYHE+/R5HoBUT11WV5O08Cr1n3YXkWVQ==", + "dev": true, + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.1", + "isarray": "0.0.1", + "string_decoder": "~0.10.x" + } + }, + "node_modules/jshint/node_modules/string_decoder": { + "version": "0.10.31", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz", + "integrity": "sha512-ev2QzSzWPYmy9GuqfIVildA4OdcGLeFZQrq5ys6RtiuF+RQQiZWr8TZNyAcuVXyQRYfEO+MsoB/1BuQVhOJuoQ==", + "dev": true + }, + "node_modules/jshint/node_modules/strip-json-comments": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-1.0.4.tgz", + "integrity": "sha512-AOPG8EBc5wAikaG1/7uFCNFJwnKOuQwFTpYBdTW6OvWHeZBQBrAA/amefHGrEiOnCPcLFZK6FUPtWVKpQVIRgg==", + "dev": true, + "bin": { + "strip-json-comments": "cli.js" + }, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/json-buffer": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/json-buffer/-/json-buffer-3.0.1.tgz", + "integrity": "sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==" + }, + "node_modules/json-parse-better-errors": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/json-parse-better-errors/-/json-parse-better-errors-1.0.2.tgz", + "integrity": "sha512-mrqyZKfX5EhL7hvqcV6WG1yYjnjeuYDzDhhcAAUrq8Po85NBQBJP+ZDUT75qZQ98IkUoBqdkExkukOU7Ts2wrw==", + "dev": true + }, + "node_modules/json-parse-even-better-errors": { + "version": "2.3.1", + "resolved": "https://registry.npmjs.org/json-parse-even-better-errors/-/json-parse-even-better-errors-2.3.1.tgz", + "integrity": "sha512-xyFwyhro/JEof6Ghe2iz2NcXoj2sloNsWr/XsERDK/oiPCfaNhl5ONfp+jQdAZRQQ0IJWNzH9zIZF7li91kh2w==", + "dev": true + }, + "node_modules/json-schema-traverse": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz", + "integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==" + }, + "node_modules/json-stringify-safe": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", + "integrity": "sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==" + }, + "node_modules/jsonata": { + "version": "1.8.6", + "resolved": "https://registry.npmjs.org/jsonata/-/jsonata-1.8.6.tgz", + "integrity": "sha512-ZH2TPYdNP2JecOl/HvrH47Xc+9imibEMQ4YqKy/F/FrM+2a6vfbGxeCX23dB9Fr6uvGwv+ghf1KxWB3iZk09wA==", + "engines": { + "node": ">= 8" + } + }, + "node_modules/jsonfile": { + "version": "6.1.0", + "resolved": "https://registry.npmjs.org/jsonfile/-/jsonfile-6.1.0.tgz", + "integrity": "sha512-5dgndWOriYSm5cnYaJNhalLNDKOqFwyDB/rr1E9ZsGciGvKPs8R2xYGCacuf3z6K1YKDz182fd+fY3cn3pMqXQ==", + "dependencies": { + "universalify": "^2.0.0" + }, + "optionalDependencies": { + "graceful-fs": "^4.1.6" + } + }, + "node_modules/just-extend": { + "version": "4.2.1", + "resolved": "https://registry.npmjs.org/just-extend/-/just-extend-4.2.1.tgz", + "integrity": "sha512-g3UB796vUFIY90VIv/WX3L2c8CS2MdWUww3CNrYmqza1Fg0DURc2K/O4YrnklBdQarSJ/y8JnJYDGc+1iumQjg==", + "dev": true + }, + "node_modules/key-list": { + "version": "0.1.4", + "resolved": "https://registry.npmjs.org/key-list/-/key-list-0.1.4.tgz", + "integrity": "sha512-DMGLZAmEoKRUHPlc772EW0i92P/WY12/oWYc2pQZb5MVGOSjYmF0BEQXbOLjbou1+/PqZ+CivwfyjaUwmyl4CQ==", + "dev": true + }, + "node_modules/keyv": { + "version": "4.5.2", + "resolved": "https://registry.npmjs.org/keyv/-/keyv-4.5.2.tgz", + "integrity": "sha512-5MHbFaKn8cNSmVW7BYnijeAVlE4cYA/SVkifVgrh7yotnfhKmjuXpDKjrABLnT0SfHWV21P8ow07OGfRrNDg8g==", + "dependencies": { + "json-buffer": "3.0.1" + } + }, + "node_modules/kind-of": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/kind-of/-/kind-of-6.0.3.tgz", + "integrity": "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/klaw": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/klaw/-/klaw-3.0.0.tgz", + "integrity": "sha512-0Fo5oir+O9jnXu5EefYbVK+mHMBeEVEy2cmctR1O1NECcCkPRreJKrS6Qt/j3KC2C148Dfo9i3pCmCMsdqGr0g==", + "dev": true, + "dependencies": { + "graceful-fs": "^4.1.9" + } + }, + "node_modules/latest-version": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/latest-version/-/latest-version-5.1.0.tgz", + "integrity": "sha512-weT+r0kTkRQdCdYCNtkMwWXQTMEswKrFBkm4ckQOMVhhqhIMI1UT2hMj+1iigIhgSZm5gTmrRXBNoGUgaTY1xA==", + "dev": true, + "dependencies": { + "package-json": "^6.3.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/lazystream": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/lazystream/-/lazystream-1.0.1.tgz", + "integrity": "sha512-b94GiNHQNy6JNTrt5w6zNyffMrNkXZb3KTkCZJb2V1xaEGCk093vkZ2jk3tpaeP33/OiXC+WvK9AxUebnf5nbw==", + "dev": true, + "dependencies": { + "readable-stream": "^2.0.5" + }, + "engines": { + "node": ">= 0.6.3" + } + }, + "node_modules/lazystream/node_modules/readable-stream": { + "version": "2.3.8", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.8.tgz", + "integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==", + "dev": true, + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.3", + "isarray": "~1.0.0", + "process-nextick-args": "~2.0.0", + "safe-buffer": "~5.1.1", + "string_decoder": "~1.1.1", + "util-deprecate": "~1.0.1" + } + }, + "node_modules/lazystream/node_modules/string_decoder": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz", + "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==", + "dev": true, + "dependencies": { + "safe-buffer": "~5.1.0" + } + }, + "node_modules/leven": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/leven/-/leven-2.1.0.tgz", + "integrity": "sha512-nvVPLpIHUxCUoRLrFqTgSxXJ614d8AgQoWl7zPe/2VadE8+1dpU3LBhowRuBAcuwruWtOdD8oYC9jDNJjXDPyA==", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/liftup": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/liftup/-/liftup-3.0.1.tgz", + "integrity": "sha512-yRHaiQDizWSzoXk3APcA71eOI/UuhEkNN9DiW2Tt44mhYzX4joFoCZlxsSOF7RyeLlfqzFLQI1ngFq3ggMPhOw==", + "dev": true, + "dependencies": { + "extend": "^3.0.2", + "findup-sync": "^4.0.0", + "fined": "^1.2.0", + "flagged-respawn": "^1.0.1", + "is-plain-object": "^2.0.4", + "object.map": "^1.0.1", + "rechoir": "^0.7.0", + "resolve": "^1.19.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/liftup/node_modules/findup-sync": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/findup-sync/-/findup-sync-4.0.0.tgz", + "integrity": "sha512-6jvvn/12IC4quLBL1KNokxC7wWTvYncaVUYSoxWw7YykPLuRrnv4qdHcSOywOI5RpkOVGeQRtWM8/q+G6W6qfQ==", + "dev": true, + "dependencies": { + "detect-file": "^1.0.0", + "is-glob": "^4.0.0", + "micromatch": "^4.0.2", + "resolve-dir": "^1.0.1" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/lines-and-columns": { + "version": "1.2.4", + "resolved": "https://registry.npmjs.org/lines-and-columns/-/lines-and-columns-1.2.4.tgz", + "integrity": "sha512-7ylylesZQ/PV29jhEDl3Ufjo6ZX7gCqJr5F7PKrqc93v7fzSymt1BpwEU8nAUXs8qzzvqhbjhK5QZg6Mt/HkBg==", + "dev": true + }, + "node_modules/linkify-it": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-3.0.3.tgz", + "integrity": "sha512-ynTsyrFSdE5oZ/O9GEf00kPngmOfVwazR5GKDq6EYfhlpFug3J2zybX56a2PRRpc9P+FuSoGNAwjlbDs9jJBPQ==", + "dev": true, + "dependencies": { + "uc.micro": "^1.0.1" + } + }, + "node_modules/livereload-js": { + "version": "2.4.0", + "resolved": "https://registry.npmjs.org/livereload-js/-/livereload-js-2.4.0.tgz", + "integrity": "sha512-XPQH8Z2GDP/Hwz2PCDrh2mth4yFejwA1OZ/81Ti3LgKyhDcEjsSsqFWZojHG0va/duGd+WyosY7eXLDoOyqcPw==", + "dev": true + }, + "node_modules/load-json-file": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/load-json-file/-/load-json-file-4.0.0.tgz", + "integrity": "sha512-Kx8hMakjX03tiGTLAIdJ+lL0htKnXjEZN6hk/tozf/WOuYGdZBJrZ+rCJRbVCugsjB3jMLn9746NsQIf5VjBMw==", + "dev": true, + "dependencies": { + "graceful-fs": "^4.1.2", + "parse-json": "^4.0.0", + "pify": "^3.0.0", + "strip-bom": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/load-json-file/node_modules/parse-json": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/parse-json/-/parse-json-4.0.0.tgz", + "integrity": "sha512-aOIos8bujGN93/8Ox/jPLh7RwVnPEysynVFE+fQZyg6jKELEHwzgKdLRFHUgXJL6kylijVSBC4BvN9OmsB48Rw==", + "dev": true, + "dependencies": { + "error-ex": "^1.3.1", + "json-parse-better-errors": "^1.0.1" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/load-json-file/node_modules/pify": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/pify/-/pify-3.0.0.tgz", + "integrity": "sha512-C3FsVNH1udSEX48gGX1xfvwTWfsYWj5U+8/uK15BGzIGrKoUpghX8hWZwa/OFnakBiiVNmBvemTJR5mcy7iPcg==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/locate-path": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/locate-path/-/locate-path-6.0.0.tgz", + "integrity": "sha512-iPZK6eYjbxRu3uB4/WZ3EsEIMJFMqAoopl3R+zuq0UjcAm/MO6KCweDgPfP3elTztoKP3KtnVHxTn2NHBSDVUw==", + "dev": true, + "dependencies": { + "p-locate": "^5.0.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/lodash": { + "version": "4.17.21", + "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz", + "integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==", + "dev": true + }, + "node_modules/lodash.camelcase": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/lodash.camelcase/-/lodash.camelcase-4.3.0.tgz", + "integrity": "sha512-TwuEnCnxbc3rAvhf/LbG7tJUDzhqXyFnv3dtzLOPgCG/hODL7WFnsbwktkD7yUV0RrreP/l1PALq/YSg6VvjlA==", + "dev": true + }, + "node_modules/lodash.clonedeep": { + "version": "4.5.0", + "resolved": "https://registry.npmjs.org/lodash.clonedeep/-/lodash.clonedeep-4.5.0.tgz", + "integrity": "sha512-H5ZhCF25riFd9uB5UCkVKo61m3S/xZk1x4wA6yp/L3RFP6Z/eHH1ymQcGLo7J3GMPfm0V/7m1tryHuGVxpqEBQ==" + }, + "node_modules/lodash.defaults": { + "version": "4.2.0", + "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz", + "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==", + "dev": true + }, + "node_modules/lodash.difference": { + "version": "4.5.0", + "resolved": "https://registry.npmjs.org/lodash.difference/-/lodash.difference-4.5.0.tgz", + "integrity": "sha512-dS2j+W26TQ7taQBGN8Lbbq04ssV3emRw4NY58WErlTO29pIqS0HmoT5aJ9+TUQ1N3G+JOZSji4eugsWwGp9yPA==", + "dev": true + }, + "node_modules/lodash.flatten": { + "version": "4.4.0", + "resolved": "https://registry.npmjs.org/lodash.flatten/-/lodash.flatten-4.4.0.tgz", + "integrity": "sha512-C5N2Z3DgnnKr0LOpv/hKCgKdb7ZZwafIrsesve6lmzvZIRZRGaZ/l6Q8+2W7NaT+ZwO3fFlSCzCzrDCFdJfZ4g==", + "dev": true + }, + "node_modules/lodash.flattendeep": { + "version": "4.4.0", + "resolved": "https://registry.npmjs.org/lodash.flattendeep/-/lodash.flattendeep-4.4.0.tgz", + "integrity": "sha512-uHaJFihxmJcEX3kT4I23ABqKKalJ/zDrDg0lsFtc1h+3uw49SIJ5beyhx5ExVRti3AvKoOJngIj7xz3oylPdWQ==", + "dev": true + }, + "node_modules/lodash.get": { + "version": "4.4.2", + "resolved": "https://registry.npmjs.org/lodash.get/-/lodash.get-4.4.2.tgz", + "integrity": "sha512-z+Uw/vLuy6gQe8cfaFWD7p0wVv8fJl3mbzXh33RS+0oW2wvUqiRXiQ69gLWSLpgB5/6sU+r6BlQR0MBILadqTQ==", + "dev": true + }, + "node_modules/lodash.isplainobject": { + "version": "4.0.6", + "resolved": "https://registry.npmjs.org/lodash.isplainobject/-/lodash.isplainobject-4.0.6.tgz", + "integrity": "sha512-oSXzaWypCMHkPC3NvBEaPHf0KsA5mvPrOPgQWDsbg8n7orZ290M0BmC/jgRZ4vcJ6DTAhjrsSYgdsW/F+MFOBA==", + "dev": true + }, + "node_modules/lodash.omit": { + "version": "4.5.0", + "resolved": "https://registry.npmjs.org/lodash.omit/-/lodash.omit-4.5.0.tgz", + "integrity": "sha512-XeqSp49hNGmlkj2EJlfrQFIzQ6lXdNro9sddtQzcJY8QaoC2GO0DT7xaIokHeyM+mIT0mPMlPvkYzg2xCuHdZg==", + "dev": true + }, + "node_modules/lodash.padend": { + "version": "4.6.1", + "resolved": "https://registry.npmjs.org/lodash.padend/-/lodash.padend-4.6.1.tgz", + "integrity": "sha512-sOQs2aqGpbl27tmCS1QNZA09Uqp01ZzWfDUoD+xzTii0E7dSQfRKcRetFwa+uXaxaqL+TKm7CgD2JdKP7aZBSw==", + "dev": true + }, + "node_modules/lodash.pick": { + "version": "4.4.0", + "resolved": "https://registry.npmjs.org/lodash.pick/-/lodash.pick-4.4.0.tgz", + "integrity": "sha512-hXt6Ul/5yWjfklSGvLQl8vM//l3FtyHZeuelpzK6mm99pNvN9yTDruNZPEJZD1oWrqo+izBmB7oUfWgcCX7s4Q==", + "dev": true + }, + "node_modules/lodash.union": { + "version": "4.6.0", + "resolved": "https://registry.npmjs.org/lodash.union/-/lodash.union-4.6.0.tgz", + "integrity": "sha512-c4pB2CdGrGdjMKYLA+XiRDO7Y0PRQbm/Gzg8qMj+QH+pFVAoTp5sBpO0odL3FjoPCGjK96p6qsP+yQoiLoOBcw==", + "dev": true + }, + "node_modules/log-symbols": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/log-symbols/-/log-symbols-4.1.0.tgz", + "integrity": "sha512-8XPvpAA8uyhfteu8pIvQxpJZ7SYYdpUivZpGy6sFsBuKRY/7rQGavedeB8aK+Zkyq6upMFVL/9AW6vOYzfRyLg==", + "dev": true, + "dependencies": { + "chalk": "^4.1.0", + "is-unicode-supported": "^0.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/lowercase-keys": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/lowercase-keys/-/lowercase-keys-2.0.0.tgz", + "integrity": "sha512-tqNXrS78oMOE73NMxK4EMLQsQowWf8jKooH9g7xPavRT706R6bkQJ6DY2Te7QukaZsulxa30wQ7bk0pm4XiHmA==", + "engines": { + "node": ">=8" + } + }, + "node_modules/lru-cache": { + "version": "4.1.5", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-4.1.5.tgz", + "integrity": "sha512-sWZlbEP2OsHNkXrMl5GYk/jKk70MBng6UU4YI/qGDYbgf6YbP4EvmqISbXCoJiRKs+1bSpFHVgQxvJ17F2li5g==", + "dependencies": { + "pseudomap": "^1.0.2", + "yallist": "^2.1.2" + } + }, + "node_modules/make-dir": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/make-dir/-/make-dir-3.1.0.tgz", + "integrity": "sha512-g3FeP20LNwhALb/6Cz6Dd4F2ngze0jz7tbzrD2wAV+o9FeNHe4rL+yK2md0J/fiSf1sa1ADhXqi5+oVwOM/eGw==", + "devOptional": true, + "dependencies": { + "semver": "^6.0.0" + }, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/make-dir/node_modules/semver": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/semver/-/semver-6.3.0.tgz", + "integrity": "sha512-b39TBaTSfV6yBrapU89p5fKekE2m/NwnDocOVruQFS1/veMgdzuPcnOM34M6CwxW8jH/lxEa5rBoDeUwu5HHTw==", + "devOptional": true, + "bin": { + "semver": "bin/semver.js" + } + }, + "node_modules/make-iterator": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/make-iterator/-/make-iterator-1.0.1.tgz", + "integrity": "sha512-pxiuXh0iVEq7VM7KMIhs5gxsfxCux2URptUQaXo4iZZJxBAzTPOLE2BumO5dbfVYq/hBJFBR/a1mFDmOx5AGmw==", + "dev": true, + "dependencies": { + "kind-of": "^6.0.2" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/map-cache": { + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/map-cache/-/map-cache-0.2.2.tgz", + "integrity": "sha512-8y/eV9QQZCiyn1SprXSrCmqJN0yNRATe+PO8ztwqrvrbdRLA3eYJF0yaR0YayLWkMbsQSKWS9N2gPcGEc4UsZg==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/markdown-it": { + "version": "12.3.2", + "resolved": "https://registry.npmjs.org/markdown-it/-/markdown-it-12.3.2.tgz", + "integrity": "sha512-TchMembfxfNVpHkbtriWltGWc+m3xszaRD0CZup7GFFhzIgQqxIfn3eGj1yZpfuflzPvfkt611B2Q/Bsk1YnGg==", + "dev": true, + "dependencies": { + "argparse": "^2.0.1", + "entities": "~2.1.0", + "linkify-it": "^3.0.1", + "mdurl": "^1.0.1", + "uc.micro": "^1.0.5" + }, + "bin": { + "markdown-it": "bin/markdown-it.js" + } + }, + "node_modules/markdown-it-anchor": { + "version": "8.6.7", + "resolved": "https://registry.npmjs.org/markdown-it-anchor/-/markdown-it-anchor-8.6.7.tgz", + "integrity": "sha512-FlCHFwNnutLgVTflOYHPW2pPcl2AACqVzExlkGQNsi4CJgqOHN7YTgDd4LuhgN1BFO3TS0vLAruV1Td6dwWPJA==", + "dev": true, + "peerDependencies": { + "@types/markdown-it": "*", + "markdown-it": "*" + } + }, + "node_modules/markdown-it/node_modules/argparse": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz", + "integrity": "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==", + "dev": true + }, + "node_modules/markdown-it/node_modules/entities": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/entities/-/entities-2.1.0.tgz", + "integrity": "sha512-hCx1oky9PFrJ611mf0ifBLBRW8lUUVRlFolb5gWRfIELabBlbp9xZvrqZLZAs+NxFnbfQoeGd8wDkygjg7U85w==", + "dev": true, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, + "node_modules/marked": { + "version": "4.0.12", + "resolved": "https://registry.npmjs.org/marked/-/marked-4.0.12.tgz", + "integrity": "sha512-hgibXWrEDNBWgGiK18j/4lkS6ihTe9sxtV4Q1OQppb/0zzyPSzoFANBa5MfsG/zgsWklmNnhm0XACZOH/0HBiQ==", + "dev": true, + "bin": { + "marked": "bin/marked.js" + }, + "engines": { + "node": ">= 12" + } + }, + "node_modules/maxmin": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/maxmin/-/maxmin-2.1.0.tgz", + "integrity": "sha512-NWlApBjW9az9qRPaeg7CX4sQBWwytqz32bIEo1PW9pRW+kBP9KLRfJO3UC+TV31EcQZEUq7eMzikC7zt3zPJcw==", + "dev": true, + "dependencies": { + "chalk": "^1.0.0", + "figures": "^1.0.1", + "gzip-size": "^3.0.0", + "pretty-bytes": "^3.0.0" + }, + "engines": { + "node": ">=0.12" + } + }, + "node_modules/maxmin/node_modules/ansi-regex": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz", + "integrity": "sha512-TIGnTpdo+E3+pCyAluZvtED5p5wCqLdezCyhPZzKPcxvFplEt4i+W7OONCKgeZFT3+y5NZZfOOS/Bdcanm1MYA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/maxmin/node_modules/ansi-styles": { + "version": "2.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-2.2.1.tgz", + "integrity": "sha512-kmCevFghRiWM7HB5zTPULl4r9bVFSWjz62MhqizDGUrq2NWuNMQyuv4tHHoKJHs69M/MF64lEcHdYIocrdWQYA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/maxmin/node_modules/chalk": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-1.1.3.tgz", + "integrity": "sha512-U3lRVLMSlsCfjqYPbLyVv11M9CPW4I728d6TCKMAOJueEeB9/8o+eSsMnxPJD+Q+K909sdESg7C+tIkoH6on1A==", + "dev": true, + "dependencies": { + "ansi-styles": "^2.2.1", + "escape-string-regexp": "^1.0.2", + "has-ansi": "^2.0.0", + "strip-ansi": "^3.0.0", + "supports-color": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/maxmin/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/maxmin/node_modules/pretty-bytes": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/pretty-bytes/-/pretty-bytes-3.0.1.tgz", + "integrity": "sha512-eb7ZAeUTgfh294cElcu51w+OTRp/6ItW758LjwJSK72LDevcuJn0P4eD71PLMDGPwwatXmAmYHTkzvpKlJE3ow==", + "dev": true, + "dependencies": { + "number-is-nan": "^1.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/maxmin/node_modules/strip-ansi": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz", + "integrity": "sha512-VhumSSbBqDTP8p2ZLKj40UjBCV4+v8bUSEpUb4KjRgWk9pbqGF4REFj6KEagidb2f/M6AzC0EmFyDNGaw9OCzg==", + "dev": true, + "dependencies": { + "ansi-regex": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/maxmin/node_modules/supports-color": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-2.0.0.tgz", + "integrity": "sha512-KKNVtd6pCYgPIKU4cp2733HWYCpplQhddZLBUryaAHou723x+FRzQ5Df824Fj+IyyuiQTRoub4SnIFfIcrp70g==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/mdurl": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/mdurl/-/mdurl-1.0.1.tgz", + "integrity": "sha512-/sKlQJCBYVY9Ers9hqzKou4H6V5UWc/M59TH2dvkt+84itfnq7uFOMLpOiOS4ujvHP4etln18fmIxA5R5fll0g==", + "dev": true + }, + "node_modules/media-typer": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-1.1.0.tgz", + "integrity": "sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/memorystore": { + "version": "1.6.7", + "resolved": "https://registry.npmjs.org/memorystore/-/memorystore-1.6.7.tgz", + "integrity": "sha512-OZnmNY/NDrKohPQ+hxp0muBcBKrzKNtHr55DbqSx9hLsYVNnomSAMRAtI7R64t3gf3ID7tHQA7mG4oL3Hu9hdw==", + "dependencies": { + "debug": "^4.3.0", + "lru-cache": "^4.0.3" + }, + "engines": { + "node": ">=0.10" + } + }, + "node_modules/memorystore/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/memorystore/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==" + }, + "node_modules/merge-descriptors": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.1.tgz", + "integrity": "sha512-cCi6g3/Zr1iqQi6ySbseM1Xvooa98N0w31jzUYrXPX2xqObmFGHJ0tQ5u74H3mVh7wLouTseZyYIq39g8cNp1w==" + }, + "node_modules/merge-source-map": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/merge-source-map/-/merge-source-map-1.1.0.tgz", + "integrity": "sha512-Qkcp7P2ygktpMPh2mCQZaf3jhN6D3Z/qVZHSdWvQ+2Ef5HgRAPBO57A77+ENm0CPx2+1Ce/MYKi3ymqdfuqibw==", + "dev": true, + "dependencies": { + "source-map": "^0.6.1" + } + }, + "node_modules/merge-source-map/node_modules/source-map": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz", + "integrity": "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/methods": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz", + "integrity": "sha512-iclAHeNqNm68zFtnZ0e+1L2yUIdvzNoauKU4WBA3VvH/vPFieF7qfRlwUZU+DA9P9bPXIS90ulxoUoCH23sV2w==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/micromatch": { + "version": "4.0.5", + "resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.5.tgz", + "integrity": "sha512-DMy+ERcEW2q8Z2Po+WNXuw3c5YaUSFjAO5GsJqfEl7UjvtIuFKO6ZrKvcItdy98dwFI2N1tg3zNIdKaQT+aNdA==", + "dev": true, + "dependencies": { + "braces": "^3.0.2", + "picomatch": "^2.3.1" + }, + "engines": { + "node": ">=8.6" + } + }, + "node_modules/mime": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/mime/-/mime-3.0.0.tgz", + "integrity": "sha512-jSCU7/VB1loIWBZe14aEYHU/+1UMEHoaO7qxCOVJOw9GgH72VAWppxNcjU+x9a2k3GSIBXNKxXQFqRvvZ7vr3A==", + "bin": { + "mime": "cli.js" + }, + "engines": { + "node": ">=10.0.0" + } + }, + "node_modules/mime-db": { + "version": "1.52.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", + "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/mime-types": { + "version": "2.1.35", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", + "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", + "dependencies": { + "mime-db": "1.52.0" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/mimic-response": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-1.0.1.tgz", + "integrity": "sha512-j5EctnkH7amfV/q5Hgmoal1g2QHFJRraOtmx0JpIqkxhBhI/lJSl1nMpQ45hVarwNETOoWEimndZ4QK0RHxuxQ==", + "engines": { + "node": ">=4" + } + }, + "node_modules/minami": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/minami/-/minami-1.2.3.tgz", + "integrity": "sha512-3f2QqqbUC1usVux0FkQMFYB73yd9JIxmHSn1dWQacizL6hOUaNu6mA3KxZ9SfiCc4qgcgq+5XP59+hP7URa1Dw==", + "dev": true + }, + "node_modules/minimatch": { + "version": "3.0.8", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.8.tgz", + "integrity": "sha512-6FsRAQsxQ61mw+qP1ZzbL9Bc78x2p5OqNgNpnoAFLTrX8n5Kxph0CsnhmKKNXTWjXqU5L0pGPR7hYk+XWZr60Q==", + "dependencies": { + "brace-expansion": "^1.1.7" + }, + "engines": { + "node": "*" + } + }, + "node_modules/minimist": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz", + "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==", + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/minipass": { + "version": "3.3.6", + "resolved": "https://registry.npmjs.org/minipass/-/minipass-3.3.6.tgz", + "integrity": "sha512-DxiNidxSEK+tHG6zOIklvNOwm3hvCrbUrdtzY74U6HKTJxvIDfOUL5W5P2Ghd3DTkhhKPYGqeNUIh5qcM4YBfw==", + "dependencies": { + "yallist": "^4.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/minipass/node_modules/yallist": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz", + "integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==" + }, + "node_modules/minizlib": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/minizlib/-/minizlib-2.1.2.tgz", + "integrity": "sha512-bAxsR8BVfj60DWXHE3u30oHzfl4G7khkSuPW+qvpd7jFRHm7dLxOjUk1EHACJ/hxLY8phGJ0YhYHZo7jil7Qdg==", + "dependencies": { + "minipass": "^3.0.0", + "yallist": "^4.0.0" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/minizlib/node_modules/yallist": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz", + "integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==" + }, + "node_modules/mkdirp": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-1.0.4.tgz", + "integrity": "sha512-vVqVZQyf3WLx2Shd0qJ9xuvqgAyKPLAiqITEtqW0oIUjzo3PePDd6fW9iFz30ef7Ysp/oiWqbhszeGWW2T6Gzw==", + "bin": { + "mkdirp": "bin/cmd.js" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/mkdirp2": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/mkdirp2/-/mkdirp2-1.0.5.tgz", + "integrity": "sha512-xOE9xbICroUDmG1ye2h4bZ8WBie9EGmACaco8K8cx6RlkJJrxGIqjGqztAI+NMhexXBcdGbSEzI6N3EJPevxZw==", + "dev": true + }, + "node_modules/mocha": { + "version": "9.2.0", + "resolved": "https://registry.npmjs.org/mocha/-/mocha-9.2.0.tgz", + "integrity": "sha512-kNn7E8g2SzVcq0a77dkphPsDSN7P+iYkqE0ZsGCYWRsoiKjOt+NvXfaagik8vuDa6W5Zw3qxe8Jfpt5qKf+6/Q==", + "dev": true, + "dependencies": { + "@ungap/promise-all-settled": "1.1.2", + "ansi-colors": "4.1.1", + "browser-stdout": "1.3.1", + "chokidar": "3.5.3", + "debug": "4.3.3", + "diff": "5.0.0", + "escape-string-regexp": "4.0.0", + "find-up": "5.0.0", + "glob": "7.2.0", + "growl": "1.10.5", + "he": "1.2.0", + "js-yaml": "4.1.0", + "log-symbols": "4.1.0", + "minimatch": "3.0.4", + "ms": "2.1.3", + "nanoid": "3.2.0", + "serialize-javascript": "6.0.0", + "strip-json-comments": "3.1.1", + "supports-color": "8.1.1", + "which": "2.0.2", + "workerpool": "6.2.0", + "yargs": "16.2.0", + "yargs-parser": "20.2.4", + "yargs-unparser": "2.0.0" + }, + "bin": { + "_mocha": "bin/_mocha", + "mocha": "bin/mocha" + }, + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/mochajs" + } + }, + "node_modules/mocha/node_modules/argparse": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz", + "integrity": "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==", + "dev": true + }, + "node_modules/mocha/node_modules/debug": { + "version": "4.3.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.3.tgz", + "integrity": "sha512-/zxw5+vh1Tfv+4Qn7a5nsbcJKPaSvCDhojn6FEl9vupwK2VCSDtEiEtqr8DFtzYFOdz63LBkxec7DYuc2jon6Q==", + "dev": true, + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/mocha/node_modules/debug/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==", + "dev": true + }, + "node_modules/mocha/node_modules/escape-string-regexp": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz", + "integrity": "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/mocha/node_modules/glob": { + "version": "7.2.0", + "resolved": "https://registry.npmjs.org/glob/-/glob-7.2.0.tgz", + "integrity": "sha512-lmLf6gtyrPq8tTjSmrO94wBeQbFR3HbLHbuyD69wuyQkImp2hWqMGB47OX65FBkPffO641IP9jWa1z4ivqG26Q==", + "dev": true, + "dependencies": { + "fs.realpath": "^1.0.0", + "inflight": "^1.0.4", + "inherits": "2", + "minimatch": "^3.0.4", + "once": "^1.3.0", + "path-is-absolute": "^1.0.0" + }, + "engines": { + "node": "*" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/mocha/node_modules/js-yaml": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.0.tgz", + "integrity": "sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA==", + "dev": true, + "dependencies": { + "argparse": "^2.0.1" + }, + "bin": { + "js-yaml": "bin/js-yaml.js" + } + }, + "node_modules/mocha/node_modules/minimatch": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz", + "integrity": "sha512-yJHVQEhyqPLUTgt9B83PXu6W3rx4MvvHvSUvToogpwoGDOUQ+yDrR0HRot+yOCdCO7u4hX3pWft6kWBBcqh0UA==", + "dev": true, + "dependencies": { + "brace-expansion": "^1.1.7" + }, + "engines": { + "node": "*" + } + }, + "node_modules/mocha/node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true + }, + "node_modules/mocha/node_modules/supports-color": { + "version": "8.1.1", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-8.1.1.tgz", + "integrity": "sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q==", + "dev": true, + "dependencies": { + "has-flag": "^4.0.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/chalk/supports-color?sponsor=1" + } + }, + "node_modules/moment": { + "version": "2.29.4", + "resolved": "https://registry.npmjs.org/moment/-/moment-2.29.4.tgz", + "integrity": "sha512-5LC9SOxjSc2HF6vO2CyuTDNivEdoz2IvyJJGj6X8DJ0eFyfszE0QiEd+iXmBvUP3WHxSjFH/vIsA0EN00cgr8w==", + "engines": { + "node": "*" + } + }, + "node_modules/moment-timezone": { + "version": "0.5.34", + "resolved": "https://registry.npmjs.org/moment-timezone/-/moment-timezone-0.5.34.tgz", + "integrity": "sha512-3zAEHh2hKUs3EXLESx/wsgw6IQdusOT8Bxm3D9UrHPQR7zlMmzwybC8zHEM1tQ4LJwP7fcxrWr8tuBg05fFCbg==", + "dependencies": { + "moment": ">= 2.9.0" + }, + "engines": { + "node": "*" + } + }, + "node_modules/mqtt": { + "version": "4.3.5", + "resolved": "https://registry.npmjs.org/mqtt/-/mqtt-4.3.5.tgz", + "integrity": "sha512-l29WGHAc0EayK1cjb6moozc+rlgK6YRCPbP3zB1CrJw84Bjk4kG9EJCXojdn4r29lA80SCqxRKq1QJ87+Xevng==", + "dependencies": { + "commist": "^1.0.0", + "concat-stream": "^2.0.0", + "debug": "^4.1.1", + "duplexify": "^4.1.1", + "help-me": "^3.0.0", + "inherits": "^2.0.3", + "lru-cache": "^6.0.0", + "minimist": "^1.2.5", + "mqtt-packet": "^6.8.0", + "number-allocator": "^1.0.9", + "pump": "^3.0.0", + "readable-stream": "^3.6.0", + "reinterval": "^1.1.0", + "rfdc": "^1.3.0", + "split2": "^3.1.0", + "ws": "^7.5.5", + "xtend": "^4.0.2" + }, + "bin": { + "mqtt": "bin/mqtt.js", + "mqtt_pub": "bin/pub.js", + "mqtt_sub": "bin/sub.js" + }, + "engines": { + "node": ">=10.0.0" + } + }, + "node_modules/mqtt-packet": { + "version": "6.10.0", + "resolved": "https://registry.npmjs.org/mqtt-packet/-/mqtt-packet-6.10.0.tgz", + "integrity": "sha512-ja8+mFKIHdB1Tpl6vac+sktqy3gA8t9Mduom1BA75cI+R9AHnZOiaBQwpGiWnaVJLDGRdNhQmFaAqd7tkKSMGA==", + "dependencies": { + "bl": "^4.0.2", + "debug": "^4.1.1", + "process-nextick-args": "^2.0.1" + } + }, + "node_modules/mqtt-packet/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/mqtt-packet/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==" + }, + "node_modules/mqtt/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/mqtt/node_modules/lru-cache": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-6.0.0.tgz", + "integrity": "sha512-Jo6dJ04CmSjuznwJSS3pUeWmd/H0ffTlkXXgwZi+eq1UCmqQwCh+eLsYOYCwY991i2Fah4h1BEMCx4qThGbsiA==", + "dependencies": { + "yallist": "^4.0.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/mqtt/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==" + }, + "node_modules/mqtt/node_modules/yallist": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz", + "integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==" + }, + "node_modules/mri": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/mri/-/mri-1.1.4.tgz", + "integrity": "sha512-6y7IjGPm8AzlvoUrwAaw1tLnUBudaS3752vcd8JtrpGGQn+rXIe63LFVHm/YMwtqAuh+LJPCFdlLYPWM1nYn6w==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/ms": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", + "integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==" + }, + "node_modules/multer": { + "version": "1.4.4", + "resolved": "https://registry.npmjs.org/multer/-/multer-1.4.4.tgz", + "integrity": "sha512-2wY2+xD4udX612aMqMcB8Ws2Voq6NIUPEtD1be6m411T4uDH/VtL9i//xvcyFlTVfRdaBsk7hV5tgrGQqhuBiw==", + "deprecated": "Multer 1.x is affected by CVE-2022-24434. This is fixed in v1.4.4-lts.1 which drops support for versions of Node.js before 6. Please upgrade to at least Node.js 6 and version 1.4.4-lts.1 of Multer. If you need support for older versions of Node.js, we are open to accepting patches that would fix the CVE on the main 1.x release line, whilst maintaining compatibility with Node.js 0.10.", + "dependencies": { + "append-field": "^1.0.0", + "busboy": "^0.2.11", + "concat-stream": "^1.5.2", + "mkdirp": "^0.5.4", + "object-assign": "^4.1.1", + "on-finished": "^2.3.0", + "type-is": "^1.6.4", + "xtend": "^4.0.0" + }, + "engines": { + "node": ">= 0.10.0" + } + }, + "node_modules/multer/node_modules/concat-stream": { + "version": "1.6.2", + "resolved": "https://registry.npmjs.org/concat-stream/-/concat-stream-1.6.2.tgz", + "integrity": "sha512-27HBghJxjiZtIk3Ycvn/4kbJk/1uZuJFfuPEns6LaEvpvG1f0hTea8lilrouyo9mVc2GWdcEZ8OLoGmSADlrCw==", + "engines": [ + "node >= 0.8" + ], + "dependencies": { + "buffer-from": "^1.0.0", + "inherits": "^2.0.3", + "readable-stream": "^2.2.2", + "typedarray": "^0.0.6" + } + }, + "node_modules/multer/node_modules/mkdirp": { + "version": "0.5.6", + "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.6.tgz", + "integrity": "sha512-FP+p8RB8OWpF3YZBCrP5gtADmtXApB5AMLn+vdyA+PyxCjrCs00mjyUozssO33cwDeT3wNGdLxJ5M//YqtHAJw==", + "dependencies": { + "minimist": "^1.2.6" + }, + "bin": { + "mkdirp": "bin/cmd.js" + } + }, + "node_modules/multer/node_modules/readable-stream": { + "version": "2.3.8", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.8.tgz", + "integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==", + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.3", + "isarray": "~1.0.0", + "process-nextick-args": "~2.0.0", + "safe-buffer": "~5.1.1", + "string_decoder": "~1.1.1", + "util-deprecate": "~1.0.1" + } + }, + "node_modules/multer/node_modules/string_decoder": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz", + "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==", + "dependencies": { + "safe-buffer": "~5.1.0" + } + }, + "node_modules/mustache": { + "version": "4.2.0", + "resolved": "https://registry.npmjs.org/mustache/-/mustache-4.2.0.tgz", + "integrity": "sha512-71ippSywq5Yb7/tVYyGbkBggbU8H3u5Rz56fH60jGFgr8uHwxs+aSKeqmluIVzM0m0kB7xQjKS6qPfd0b2ZoqQ==", + "bin": { + "mustache": "bin/mustache" + } + }, + "node_modules/mute-stream": { + "version": "0.0.8", + "resolved": "https://registry.npmjs.org/mute-stream/-/mute-stream-0.0.8.tgz", + "integrity": "sha512-nnbWWOkoWyUsTjKrhgD0dcz22mdkSnpYqbEjIm2nhwhuxlSkpywJmBo8h0ZqJdkp73mb90SssHkN4rsRaBAfAA==" + }, + "node_modules/nanoid": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.2.0.tgz", + "integrity": "sha512-fmsZYa9lpn69Ad5eDn7FMcnnSR+8R34W9qJEijxYhTbfOWzr22n1QxCMzXLK+ODyW2973V3Fux959iQoUxzUIA==", + "dev": true, + "bin": { + "nanoid": "bin/nanoid.cjs" + }, + "engines": { + "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" + } + }, + "node_modules/negotiator": { + "version": "0.6.3", + "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz", + "integrity": "sha512-+EUsqGPLsM+j/zdChZjsnX51g4XrHFOIXwfnCVPGlQk/k5giakcKsuxCObBRu6DSm9opw/O6slWbJdghQM4bBg==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/neo-async": { + "version": "2.6.2", + "resolved": "https://registry.npmjs.org/neo-async/-/neo-async-2.6.2.tgz", + "integrity": "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw==", + "dev": true + }, + "node_modules/nested-error-stacks": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/nested-error-stacks/-/nested-error-stacks-2.1.1.tgz", + "integrity": "sha512-9iN1ka/9zmX1ZvLV9ewJYEk9h7RyRRtqdK0woXcqohu8EWIerfPUjYJPg0ULy0UqP7cslmdGc8xKDJcojlKiaw==", + "dev": true + }, + "node_modules/nise": { + "version": "5.1.4", + "resolved": "https://registry.npmjs.org/nise/-/nise-5.1.4.tgz", + "integrity": "sha512-8+Ib8rRJ4L0o3kfmyVCL7gzrohyDe0cMFTBa2d364yIrEGMEoetznKJx899YxjybU6bL9SQkYPSBBs1gyYs8Xg==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^2.0.0", + "@sinonjs/fake-timers": "^10.0.2", + "@sinonjs/text-encoding": "^0.7.1", + "just-extend": "^4.0.2", + "path-to-regexp": "^1.7.0" + } + }, + "node_modules/nise/node_modules/@sinonjs/commons": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@sinonjs/commons/-/commons-2.0.0.tgz", + "integrity": "sha512-uLa0j859mMrg2slwQYdO/AkrOfmH+X6LTVmNTS9CqexuE2IvVORIkSpJLqePAbEnKJ77aMmCwr1NUZ57120Xcg==", + "dev": true, + "dependencies": { + "type-detect": "4.0.8" + } + }, + "node_modules/nise/node_modules/@sinonjs/fake-timers": { + "version": "10.0.2", + "resolved": "https://registry.npmjs.org/@sinonjs/fake-timers/-/fake-timers-10.0.2.tgz", + "integrity": "sha512-SwUDyjWnah1AaNl7kxsa7cfLhlTYoiyhDAIgyh+El30YvXs/o7OLXpYH88Zdhyx9JExKrmHDJ+10bwIcY80Jmw==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^2.0.0" + } + }, + "node_modules/nise/node_modules/isarray": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", + "integrity": "sha512-D2S+3GLxWH+uhrNEcoh/fnmYeP8E8/zHl644d/jdA0g2uyXvy3sb0qxotE+ne0LtccHknQzWwZEzhak7oJ0COQ==", + "dev": true + }, + "node_modules/nise/node_modules/path-to-regexp": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-1.8.0.tgz", + "integrity": "sha512-n43JRhlUKUAlibEJhPeir1ncUID16QnEjNpwzNdO3Lm4ywrBpBZ5oLD0I6br9evr1Y9JTqwRtAh7JLoOzAQdVA==", + "dev": true, + "dependencies": { + "isarray": "0.0.1" + } + }, + "node_modules/node-addon-api": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-3.2.1.tgz", + "integrity": "sha512-mmcei9JghVNDYydghQmeDX8KoAm0FAiYyIcUt/N4nhyAipB17pllZQDOJD2fotxABnt4Mdz+dKTO7eftLg4d0A==", + "optional": true + }, + "node_modules/node-fetch": { + "version": "2.6.9", + "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.6.9.tgz", + "integrity": "sha512-DJm/CJkZkRjKKj4Zi4BsKVZh3ValV5IR5s7LVZnW+6YMh0W1BfNA8XSs6DLMGYlId5F3KnA70uu2qepcR08Qqg==", + "optional": true, + "dependencies": { + "whatwg-url": "^5.0.0" + }, + "engines": { + "node": "4.x || >=6.0.0" + }, + "peerDependencies": { + "encoding": "^0.1.0" + }, + "peerDependenciesMeta": { + "encoding": { + "optional": true + } + } + }, + "node_modules/node-red": { + "version": "2.2.3", + "resolved": "https://registry.npmjs.org/node-red/-/node-red-2.2.3.tgz", + "integrity": "sha512-sfOsWONOF5TyNApXv6AkLzwfxq+C+3lyUrXRvyuQxWt1dSGvfg417dbEx4WW0kxwREGZ47TKtLejnoTPnV5adQ==", + "dependencies": { + "@node-red/editor-api": "2.2.3", + "@node-red/nodes": "2.2.3", + "@node-red/runtime": "2.2.3", + "@node-red/util": "2.2.3", + "basic-auth": "2.0.1", + "bcryptjs": "2.4.3", + "express": "4.17.2", + "fs-extra": "10.0.0", + "node-red-admin": "^2.2.3", + "nopt": "5.0.0", + "semver": "7.3.5" + }, + "bin": { + "node-red": "red.js", + "node-red-pi": "bin/node-red-pi" + }, + "engines": { + "node": ">=12" + }, + "optionalDependencies": { + "bcrypt": "5.0.1" + } + }, + "node_modules/node-red-admin": { + "version": "2.2.4", + "resolved": "https://registry.npmjs.org/node-red-admin/-/node-red-admin-2.2.4.tgz", + "integrity": "sha512-DlJpMFopqBNj10k5rGGI9ZNBi+whAIS+IHrSZH1xllfuJKZxQBZgR+o+rJeufDyc0OBRgHRqmX776HrBrlDtMA==", + "dependencies": { + "ansi-colors": "^4.1.1", + "axios": "0.27.0", + "bcryptjs": "^2.4.3", + "cli-table": "^0.3.11", + "enquirer": "^2.3.6", + "minimist": "^1.2.6", + "mustache": "^4.2.0", + "read": "^1.0.7" + }, + "bin": { + "node-red-admin": "node-red-admin.js" + }, + "engines": { + "node": ">=12" + }, + "optionalDependencies": { + "bcrypt": "5.0.1" + } + }, + "node_modules/node-red-contrib-pythonshell": { + "version": "1.5.4", + "resolved": "git+ssh://git@github.com/namgk/node-red-contrib-pythonshell.git#6bcc7df0003600bde8d688ece9c8d20caa00992f", + "license": "ISC" + }, + "node_modules/node-red-node-test-helper": { + "version": "0.2.7", + "resolved": "https://registry.npmjs.org/node-red-node-test-helper/-/node-red-node-test-helper-0.2.7.tgz", + "integrity": "sha512-OanSQ1hrsigHVtMjL/cuhtjxhTdRBXxd3IALJC9eg0WOHRF75ZI7RYhFWqqOsvQ++BwmNj8ki1S49D8cZyZTWA==", + "dev": true, + "dependencies": { + "body-parser": "1.19.0", + "express": "4.17.1", + "read-pkg-up": "7.0.1", + "semver": "7.3.4", + "should": "^13.2.3", + "should-sinon": "0.0.6", + "sinon": "9.2.4", + "stoppable": "1.1.0", + "supertest": "4.0.2" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/node-red-node-test-helper/node_modules/@sinonjs/fake-timers": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/@sinonjs/fake-timers/-/fake-timers-6.0.1.tgz", + "integrity": "sha512-MZPUxrmFubI36XS1DI3qmI0YdN1gks62JtFZvxR67ljjSNCeK6U08Zx4msEWOXuofgqUt6zPHSi1H9fbjR/NRA==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^1.7.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/@sinonjs/samsam": { + "version": "5.3.1", + "resolved": "https://registry.npmjs.org/@sinonjs/samsam/-/samsam-5.3.1.tgz", + "integrity": "sha512-1Hc0b1TtyfBu8ixF/tpfSHTVWKwCBLY4QJbkgnE7HcwyvT2xArDxb4K7dMgqRm3szI+LJbzmW/s4xxEhv6hwDg==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^1.6.0", + "lodash.get": "^4.4.2", + "type-detect": "^4.0.8" + } + }, + "node_modules/node-red-node-test-helper/node_modules/body-parser": { + "version": "1.19.0", + "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.19.0.tgz", + "integrity": "sha512-dhEPs72UPbDnAQJ9ZKMNTP6ptJaionhP5cBb541nXPlW60Jepo9RV/a4fX4XWW9CuFNK22krhrj1+rgzifNCsw==", + "dev": true, + "dependencies": { + "bytes": "3.1.0", + "content-type": "~1.0.4", + "debug": "2.6.9", + "depd": "~1.1.2", + "http-errors": "1.7.2", + "iconv-lite": "0.4.24", + "on-finished": "~2.3.0", + "qs": "6.7.0", + "raw-body": "2.4.0", + "type-is": "~1.6.17" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/node-red-node-test-helper/node_modules/bytes": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.0.tgz", + "integrity": "sha512-zauLjrfCG+xvoyaqLoV8bLVXXNGC4JqlxFCutSDWA6fJrTo2ZuvLYTqZ7aHBLZSMOopbzwv8f+wZcVzfVTI2Dg==", + "dev": true, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/node-red-node-test-helper/node_modules/content-disposition": { + "version": "0.5.3", + "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.3.tgz", + "integrity": "sha512-ExO0774ikEObIAEV9kDo50o+79VCUdEB6n6lzKgGwupcVeRlhrj3qGAfwq8G6uBJjkqLrhT0qEYFcWng8z1z0g==", + "dev": true, + "dependencies": { + "safe-buffer": "5.1.2" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/node-red-node-test-helper/node_modules/cookie": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.4.0.tgz", + "integrity": "sha512-+Hp8fLp57wnUSt0tY0tHEXh4voZRDnoIrZPqlo3DPiI4y9lwg/jqx+1Om94/W6ZaPDOUbnjOt/99w66zk+l1Xg==", + "dev": true, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/node-red-node-test-helper/node_modules/diff": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/diff/-/diff-4.0.2.tgz", + "integrity": "sha512-58lmxKSA4BNyLz+HHMUzlOEpg09FV+ev6ZMe3vJihgdxzgcwZ8VoEEPmALCZG9LmqfVoNMMKpttIYTVG6uDY7A==", + "dev": true, + "engines": { + "node": ">=0.3.1" + } + }, + "node_modules/node-red-node-test-helper/node_modules/express": { + "version": "4.17.1", + "resolved": "https://registry.npmjs.org/express/-/express-4.17.1.tgz", + "integrity": "sha512-mHJ9O79RqluphRrcw2X/GTh3k9tVv8YcoyY4Kkh4WDMUYKRZUq0h1o0w2rrrxBqM7VoeUVqgb27xlEMXTnYt4g==", + "dev": true, + "dependencies": { + "accepts": "~1.3.7", + "array-flatten": "1.1.1", + "body-parser": "1.19.0", + "content-disposition": "0.5.3", + "content-type": "~1.0.4", + "cookie": "0.4.0", + "cookie-signature": "1.0.6", + "debug": "2.6.9", + "depd": "~1.1.2", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "finalhandler": "~1.1.2", + "fresh": "0.5.2", + "merge-descriptors": "1.0.1", + "methods": "~1.1.2", + "on-finished": "~2.3.0", + "parseurl": "~1.3.3", + "path-to-regexp": "0.1.7", + "proxy-addr": "~2.0.5", + "qs": "6.7.0", + "range-parser": "~1.2.1", + "safe-buffer": "5.1.2", + "send": "0.17.1", + "serve-static": "1.14.1", + "setprototypeof": "1.1.1", + "statuses": "~1.5.0", + "type-is": "~1.6.18", + "utils-merge": "1.0.1", + "vary": "~1.1.2" + }, + "engines": { + "node": ">= 0.10.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/form-data": { + "version": "2.5.1", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-2.5.1.tgz", + "integrity": "sha512-m21N3WOmEEURgk6B9GLOE4RuWOFf28Lhh9qGYeNlGq4VDXUlJy2th2slBNU8Gp8EzloYZOibZJ7t5ecIrFSjVA==", + "dev": true, + "dependencies": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.6", + "mime-types": "^2.1.12" + }, + "engines": { + "node": ">= 0.12" + } + }, + "node_modules/node-red-node-test-helper/node_modules/formidable": { + "version": "1.2.6", + "resolved": "https://registry.npmjs.org/formidable/-/formidable-1.2.6.tgz", + "integrity": "sha512-KcpbcpuLNOwrEjnbpMC0gS+X8ciDoZE1kkqzat4a8vrprf+s9pKNQ/QIwWfbfs4ltgmFl3MD177SNTkve3BwGQ==", + "deprecated": "Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau", + "dev": true, + "funding": { + "url": "https://ko-fi.com/tunnckoCore/commissions" + } + }, + "node_modules/node-red-node-test-helper/node_modules/http-errors": { + "version": "1.7.2", + "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.7.2.tgz", + "integrity": "sha512-uUQBt3H/cSIVfch6i1EuPNy/YsRSOUBXTVfZ+yR7Zjez3qjBz6i9+i4zjNaoqcoFVI4lQJ5plg63TvGfRSDCRg==", + "dev": true, + "dependencies": { + "depd": "~1.1.2", + "inherits": "2.0.3", + "setprototypeof": "1.1.1", + "statuses": ">= 1.5.0 < 2", + "toidentifier": "1.0.0" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/node-red-node-test-helper/node_modules/iconv-lite": { + "version": "0.4.24", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz", + "integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==", + "dev": true, + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/inherits": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz", + "integrity": "sha512-x00IRNXNy63jwGkJmzPigoySHbaqpNuzKbBOmzK+g2OdZpQ9w+sxCN+VSB3ja7IAge2OP2qpfxTjeNcyjmW1uw==", + "dev": true + }, + "node_modules/node-red-node-test-helper/node_modules/lru-cache": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-6.0.0.tgz", + "integrity": "sha512-Jo6dJ04CmSjuznwJSS3pUeWmd/H0ffTlkXXgwZi+eq1UCmqQwCh+eLsYOYCwY991i2Fah4h1BEMCx4qThGbsiA==", + "dev": true, + "dependencies": { + "yallist": "^4.0.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/node-red-node-test-helper/node_modules/mime": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz", + "integrity": "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==", + "dev": true, + "bin": { + "mime": "cli.js" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/node-red-node-test-helper/node_modules/ms": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.1.tgz", + "integrity": "sha512-tgp+dl5cGk28utYktBsrFqA7HKgrhgPsg6Z/EfhWI4gl1Hwq8B/GmY/0oXZ6nF8hDVesS/FpnYaD/kOWhYQvyg==", + "dev": true + }, + "node_modules/node-red-node-test-helper/node_modules/nise": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/nise/-/nise-4.1.0.tgz", + "integrity": "sha512-eQMEmGN/8arp0xsvGoQ+B1qvSkR73B1nWSCh7nOt5neMCtwcQVYQGdzQMhcNscktTsWB54xnlSQFzOAPJD8nXA==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^1.7.0", + "@sinonjs/fake-timers": "^6.0.0", + "@sinonjs/text-encoding": "^0.7.1", + "just-extend": "^4.0.2", + "path-to-regexp": "^1.7.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/nise/node_modules/isarray": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", + "integrity": "sha512-D2S+3GLxWH+uhrNEcoh/fnmYeP8E8/zHl644d/jdA0g2uyXvy3sb0qxotE+ne0LtccHknQzWwZEzhak7oJ0COQ==", + "dev": true + }, + "node_modules/node-red-node-test-helper/node_modules/nise/node_modules/path-to-regexp": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-1.8.0.tgz", + "integrity": "sha512-n43JRhlUKUAlibEJhPeir1ncUID16QnEjNpwzNdO3Lm4ywrBpBZ5oLD0I6br9evr1Y9JTqwRtAh7JLoOzAQdVA==", + "dev": true, + "dependencies": { + "isarray": "0.0.1" + } + }, + "node_modules/node-red-node-test-helper/node_modules/qs": { + "version": "6.7.0", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.7.0.tgz", + "integrity": "sha512-VCdBRNFTX1fyE7Nb6FYoURo/SPe62QCaAyzJvUjwRaIsc+NePBEniHlvxFmmX56+HZphIGtV0XeCirBtpDrTyQ==", + "dev": true, + "engines": { + "node": ">=0.6" + } + }, + "node_modules/node-red-node-test-helper/node_modules/raw-body": { + "version": "2.4.0", + "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.4.0.tgz", + "integrity": "sha512-4Oz8DUIwdvoa5qMJelxipzi/iJIi40O5cGV1wNYp5hvZP8ZN0T+jiNkL0QepXs+EsQ9XJ8ipEDoiH70ySUJP3Q==", + "dev": true, + "dependencies": { + "bytes": "3.1.0", + "http-errors": "1.7.2", + "iconv-lite": "0.4.24", + "unpipe": "1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/node-red-node-test-helper/node_modules/readable-stream": { + "version": "2.3.8", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.8.tgz", + "integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==", + "dev": true, + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.3", + "isarray": "~1.0.0", + "process-nextick-args": "~2.0.0", + "safe-buffer": "~5.1.1", + "string_decoder": "~1.1.1", + "util-deprecate": "~1.0.1" + } + }, + "node_modules/node-red-node-test-helper/node_modules/semver": { + "version": "7.3.4", + "resolved": "https://registry.npmjs.org/semver/-/semver-7.3.4.tgz", + "integrity": "sha512-tCfb2WLjqFAtXn4KEdxIhalnRtoKFN7nAwj0B3ZXCbQloV2tq5eDbcTmT68JJD3nRJq24/XgxtQKFIpQdtvmVw==", + "dev": true, + "dependencies": { + "lru-cache": "^6.0.0" + }, + "bin": { + "semver": "bin/semver.js" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/node-red-node-test-helper/node_modules/send": { + "version": "0.17.1", + "resolved": "https://registry.npmjs.org/send/-/send-0.17.1.tgz", + "integrity": "sha512-BsVKsiGcQMFwT8UxypobUKyv7irCNRHk1T0G680vk88yf6LBByGcZJOTJCrTP2xVN6yI+XjPJcNuE3V4fT9sAg==", + "dev": true, + "dependencies": { + "debug": "2.6.9", + "depd": "~1.1.2", + "destroy": "~1.0.4", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "fresh": "0.5.2", + "http-errors": "~1.7.2", + "mime": "1.6.0", + "ms": "2.1.1", + "on-finished": "~2.3.0", + "range-parser": "~1.2.1", + "statuses": "~1.5.0" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/serve-static": { + "version": "1.14.1", + "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.14.1.tgz", + "integrity": "sha512-JMrvUwE54emCYWlTI+hGrGv5I8dEwmco/00EvkzIIsR7MqrHonbD9pO2MOfFnpFntl7ecpZs+3mW+XbQZu9QCg==", + "dev": true, + "dependencies": { + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "parseurl": "~1.3.3", + "send": "0.17.1" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/setprototypeof": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.1.1.tgz", + "integrity": "sha512-JvdAWfbXeIGaZ9cILp38HntZSFSo3mWg6xGcJJsd+d4aRMOqauag1C63dJfDw7OaMYwEbHMOxEZ1lqVRYP2OAw==", + "dev": true + }, + "node_modules/node-red-node-test-helper/node_modules/sinon": { + "version": "9.2.4", + "resolved": "https://registry.npmjs.org/sinon/-/sinon-9.2.4.tgz", + "integrity": "sha512-zljcULZQsJxVra28qIAL6ow1Z9tpattkCTEJR4RBP3TGc00FcttsP5pK284Nas5WjMZU5Yzy3kAIp3B3KRf5Yg==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^1.8.1", + "@sinonjs/fake-timers": "^6.0.1", + "@sinonjs/samsam": "^5.3.1", + "diff": "^4.0.2", + "nise": "^4.0.4", + "supports-color": "^7.1.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/sinon" + } + }, + "node_modules/node-red-node-test-helper/node_modules/string_decoder": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz", + "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==", + "dev": true, + "dependencies": { + "safe-buffer": "~5.1.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/superagent": { + "version": "3.8.3", + "resolved": "https://registry.npmjs.org/superagent/-/superagent-3.8.3.tgz", + "integrity": "sha512-GLQtLMCoEIK4eDv6OGtkOoSMt3D+oq0y3dsxMuYuDvaNUvuT8eFBuLmfR0iYYzHC1e8hpzC6ZsxbuP6DIalMFA==", + "deprecated": "Please upgrade to v7.0.2+ of superagent. We have fixed numerous issues with streams, form-data, attach(), filesystem errors not bubbling up (ENOENT on attach()), and all tests are now passing. See the releases tab for more information at .", + "dev": true, + "dependencies": { + "component-emitter": "^1.2.0", + "cookiejar": "^2.1.0", + "debug": "^3.1.0", + "extend": "^3.0.0", + "form-data": "^2.3.1", + "formidable": "^1.2.0", + "methods": "^1.1.1", + "mime": "^1.4.1", + "qs": "^6.5.1", + "readable-stream": "^2.3.5" + }, + "engines": { + "node": ">= 4.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/superagent/node_modules/debug": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/debug/-/debug-3.2.7.tgz", + "integrity": "sha512-CFjzYYAi4ThfiQvizrFQevTTXHtnCqWfe7x1AhgEscTz6ZbLbfoLRLPugTQyBth6f8ZERVUSyWHFD/7Wu4t1XQ==", + "dev": true, + "dependencies": { + "ms": "^2.1.1" + } + }, + "node_modules/node-red-node-test-helper/node_modules/supertest": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/supertest/-/supertest-4.0.2.tgz", + "integrity": "sha512-1BAbvrOZsGA3YTCWqbmh14L0YEq0EGICX/nBnfkfVJn7SrxQV1I3pMYjSzG9y/7ZU2V9dWqyqk2POwxlb09duQ==", + "dev": true, + "dependencies": { + "methods": "^1.1.2", + "superagent": "^3.8.3" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/node-red-node-test-helper/node_modules/toidentifier": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.0.tgz", + "integrity": "sha512-yaOH/Pk/VEhBWWTlhI+qXxDFXlejDGcQipMlyxda9nthulaxLZUNcUqFxokp0vcYnvteJln5FNQDRrxj3YcbVw==", + "dev": true, + "engines": { + "node": ">=0.6" + } + }, + "node_modules/node-red-node-test-helper/node_modules/yallist": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz", + "integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==", + "dev": true + }, + "node_modules/nodemon": { + "version": "2.0.15", + "resolved": "https://registry.npmjs.org/nodemon/-/nodemon-2.0.15.tgz", + "integrity": "sha512-gdHMNx47Gw7b3kWxJV64NI+Q5nfl0y5DgDbiVtShiwa7Z0IZ07Ll4RLFo6AjrhzMtoEZn5PDE3/c2AbVsiCkpA==", + "dev": true, + "hasInstallScript": true, + "dependencies": { + "chokidar": "^3.5.2", + "debug": "^3.2.7", + "ignore-by-default": "^1.0.1", + "minimatch": "^3.0.4", + "pstree.remy": "^1.1.8", + "semver": "^5.7.1", + "supports-color": "^5.5.0", + "touch": "^3.1.0", + "undefsafe": "^2.0.5", + "update-notifier": "^5.1.0" + }, + "bin": { + "nodemon": "bin/nodemon.js" + }, + "engines": { + "node": ">=8.10.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/nodemon" + } + }, + "node_modules/nodemon/node_modules/debug": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/debug/-/debug-3.2.7.tgz", + "integrity": "sha512-CFjzYYAi4ThfiQvizrFQevTTXHtnCqWfe7x1AhgEscTz6ZbLbfoLRLPugTQyBth6f8ZERVUSyWHFD/7Wu4t1XQ==", + "dev": true, + "dependencies": { + "ms": "^2.1.1" + } + }, + "node_modules/nodemon/node_modules/has-flag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", + "integrity": "sha512-sKJf1+ceQBr4SMkvQnBDNDtf4TXpVhVGateu0t918bl30FnbE2m4vNLX+VWe/dpjlb+HugGYzW7uQXH98HPEYw==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/nodemon/node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true + }, + "node_modules/nodemon/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/nodemon/node_modules/supports-color": { + "version": "5.5.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz", + "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==", + "dev": true, + "dependencies": { + "has-flag": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/nopt": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/nopt/-/nopt-5.0.0.tgz", + "integrity": "sha512-Tbj67rffqceeLpcRXrT7vKAN8CwfPeIBgM7E6iBkmKLV7bEMwpGgYLGv0jACUsECaa/vuxP0IjEont6umdMgtQ==", + "dependencies": { + "abbrev": "1" + }, + "bin": { + "nopt": "bin/nopt.js" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/normalize-package-data": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/normalize-package-data/-/normalize-package-data-2.5.0.tgz", + "integrity": "sha512-/5CMN3T0R4XTj4DcGaexo+roZSdSFW/0AOOTROrjxzCG1wrWXEsGbRKevjlIL+ZDE4sZlJr5ED4YW0yqmkK+eA==", + "dev": true, + "dependencies": { + "hosted-git-info": "^2.1.4", + "resolve": "^1.10.0", + "semver": "2 || 3 || 4 || 5", + "validate-npm-package-license": "^3.0.1" + } + }, + "node_modules/normalize-package-data/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/normalize-path": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz", + "integrity": "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/normalize-url": { + "version": "6.1.0", + "resolved": "https://registry.npmjs.org/normalize-url/-/normalize-url-6.1.0.tgz", + "integrity": "sha512-DlL+XwOy3NxAQ8xuC0okPgK46iuVNAK01YN7RueYBqqFeGsBjV9XmCAzAdgt+667bCl5kPh9EqKKDwnaPG1I7A==", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/npmlog": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/npmlog/-/npmlog-5.0.1.tgz", + "integrity": "sha512-AqZtDUWOMKs1G/8lwylVjrdYgqA4d9nu8hc+0gzRxlDb1I10+FHBGMXs6aiQHFdCUUlqH99MUMuLfzWDNDtfxw==", + "optional": true, + "dependencies": { + "are-we-there-yet": "^2.0.0", + "console-control-strings": "^1.1.0", + "gauge": "^3.0.0", + "set-blocking": "^2.0.0" + } + }, + "node_modules/nth-check": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz", + "integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==", + "dependencies": { + "boolbase": "^1.0.0" + }, + "funding": { + "url": "https://github.com/fb55/nth-check?sponsor=1" + } + }, + "node_modules/number-allocator": { + "version": "1.0.14", + "resolved": "https://registry.npmjs.org/number-allocator/-/number-allocator-1.0.14.tgz", + "integrity": "sha512-OrL44UTVAvkKdOdRQZIJpLkAdjXGTRda052sN4sO77bKEzYYqWKMBjQvrJFzqygI99gL6Z4u2xctPW1tB8ErvA==", + "dependencies": { + "debug": "^4.3.1", + "js-sdsl": "4.3.0" + } + }, + "node_modules/number-allocator/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/number-allocator/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==" + }, + "node_modules/number-is-nan": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/number-is-nan/-/number-is-nan-1.0.1.tgz", + "integrity": "sha512-4jbtZXNAsfZbAHiiqjLPBiCl16dES1zI4Hpzzxw61Tk+loF+sBDBKx1ICKKKwIqQ7M0mFn1TmkN7euSncWgHiQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/nyc": { + "version": "14.1.1", + "resolved": "https://registry.npmjs.org/nyc/-/nyc-14.1.1.tgz", + "integrity": "sha512-OI0vm6ZGUnoGZv/tLdZ2esSVzDwUC88SNs+6JoSOMVxA+gKMB8Tk7jBwgemLx4O40lhhvZCVw1C+OYLOBOPXWw==", + "dev": true, + "dependencies": { + "archy": "^1.0.0", + "caching-transform": "^3.0.2", + "convert-source-map": "^1.6.0", + "cp-file": "^6.2.0", + "find-cache-dir": "^2.1.0", + "find-up": "^3.0.0", + "foreground-child": "^1.5.6", + "glob": "^7.1.3", + "istanbul-lib-coverage": "^2.0.5", + "istanbul-lib-hook": "^2.0.7", + "istanbul-lib-instrument": "^3.3.0", + "istanbul-lib-report": "^2.0.8", + "istanbul-lib-source-maps": "^3.0.6", + "istanbul-reports": "^2.2.4", + "js-yaml": "^3.13.1", + "make-dir": "^2.1.0", + "merge-source-map": "^1.1.0", + "resolve-from": "^4.0.0", + "rimraf": "^2.6.3", + "signal-exit": "^3.0.2", + "spawn-wrap": "^1.4.2", + "test-exclude": "^5.2.3", + "uuid": "^3.3.2", + "yargs": "^13.2.2", + "yargs-parser": "^13.0.0" + }, + "bin": { + "nyc": "bin/nyc.js" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/ansi-regex": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-4.1.1.tgz", + "integrity": "sha512-ILlv4k/3f6vfQ4OoP2AGvirOktlQ98ZEL1k9FaQjxa3L1abBgbuTDAdPOpvbGncC0BTVQrl+OM8xZGK6tWXt7g==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/ansi-styles": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz", + "integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==", + "dev": true, + "dependencies": { + "color-convert": "^1.9.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/nyc/node_modules/cliui": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/cliui/-/cliui-5.0.0.tgz", + "integrity": "sha512-PYeGSEmmHM6zvoef2w8TPzlrnNpXIjTipYK780YswmIP9vjxmd6Y2a3CB2Ks6/AU8NHjZugXvo8w3oWM2qnwXA==", + "dev": true, + "dependencies": { + "string-width": "^3.1.0", + "strip-ansi": "^5.2.0", + "wrap-ansi": "^5.1.0" + } + }, + "node_modules/nyc/node_modules/color-convert": { + "version": "1.9.3", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.3.tgz", + "integrity": "sha512-QfAUtd+vFdAtFQcC8CCyYt1fYWxSqAiK2cSD6zDB8N3cpsEBAvRxp9zOGg6G/SHHJYAT88/az/IuDGALsNVbGg==", + "dev": true, + "dependencies": { + "color-name": "1.1.3" + } + }, + "node_modules/nyc/node_modules/color-name": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz", + "integrity": "sha512-72fSenhMw2HZMTVHeCA9KCmpEIbzWiQsjN+BHcBbS9vr1mtt+vJjPdksIBNUmKAW8TFUDPJK5SUU3QhE9NEXDw==", + "dev": true + }, + "node_modules/nyc/node_modules/decamelize": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/decamelize/-/decamelize-1.2.0.tgz", + "integrity": "sha512-z2S+W9X73hAUUki+N+9Za2lBlun89zigOyGrsax+KUQ6wKW4ZoWpEYBkGhQjwAjjDCkWxhY0VKEhk8wzY7F5cA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/nyc/node_modules/emoji-regex": { + "version": "7.0.3", + "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-7.0.3.tgz", + "integrity": "sha512-CwBLREIQ7LvYFB0WyRvwhq5N5qPhc6PMjD6bYggFlI5YyDgl+0vxq5VHbMOFqLg7hfWzmu8T5Z1QofhmTIhItA==", + "dev": true + }, + "node_modules/nyc/node_modules/find-up": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/find-up/-/find-up-3.0.0.tgz", + "integrity": "sha512-1yD6RmLI1XBfxugvORwlck6f75tYL+iR0jqwsOrOxMZyGYqUuDhJ0l4AXdO1iX/FTs9cBAMEk1gWSEx1kSbylg==", + "dev": true, + "dependencies": { + "locate-path": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/is-fullwidth-code-point": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-2.0.0.tgz", + "integrity": "sha512-VHskAKYM8RfSFXwee5t5cbN5PZeq1Wrh6qd5bkyiXIf6UQcN6w/A0eXM9r6t8d+GYOh+o6ZhiEnb88LN/Y8m2w==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/nyc/node_modules/locate-path": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/locate-path/-/locate-path-3.0.0.tgz", + "integrity": "sha512-7AO748wWnIhNqAuaty2ZWHkQHRSNfPVIsPIfwEOWO22AmaoVrWavlOcMR5nzTLNYvp36X220/maaRsrec1G65A==", + "dev": true, + "dependencies": { + "p-locate": "^3.0.0", + "path-exists": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/make-dir": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/make-dir/-/make-dir-2.1.0.tgz", + "integrity": "sha512-LS9X+dc8KLxXCb8dni79fLIIUA5VyZoyjSMCwTluaXA0o27cCK0bhXkpgw+sTXVpPy/lSO57ilRixqk0vDmtRA==", + "dev": true, + "dependencies": { + "pify": "^4.0.1", + "semver": "^5.6.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/p-limit": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/p-limit/-/p-limit-2.3.0.tgz", + "integrity": "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w==", + "dev": true, + "dependencies": { + "p-try": "^2.0.0" + }, + "engines": { + "node": ">=6" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/nyc/node_modules/p-locate": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/p-locate/-/p-locate-3.0.0.tgz", + "integrity": "sha512-x+12w/To+4GFfgJhBEpiDcLozRJGegY+Ei7/z0tSLkMmxGZNybVMSfWj9aJn8Z5Fc7dBUNJOOVgPv2H7IwulSQ==", + "dev": true, + "dependencies": { + "p-limit": "^2.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/path-exists": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-3.0.0.tgz", + "integrity": "sha512-bpC7GYwiDYQ4wYLe+FA8lhRjhQCMcQGuSgGGqDkg/QerRWw9CmGRT0iSOVRSZJ29NMLZgIzqaljJ63oaL4NIJQ==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/nyc/node_modules/rimraf": { + "version": "2.7.1", + "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-2.7.1.tgz", + "integrity": "sha512-uWjbaKIK3T1OSVptzX7Nl6PvQ3qAGtKEtVRjRuazjfL3Bx5eI409VZSqgND+4UNnmzLVdPj9FqFJNPqBZFve4w==", + "dev": true, + "dependencies": { + "glob": "^7.1.3" + }, + "bin": { + "rimraf": "bin.js" + } + }, + "node_modules/nyc/node_modules/semver": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz", + "integrity": "sha512-sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==", + "dev": true, + "bin": { + "semver": "bin/semver" + } + }, + "node_modules/nyc/node_modules/string-width": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/string-width/-/string-width-3.1.0.tgz", + "integrity": "sha512-vafcv6KjVZKSgz06oM/H6GDBrAtz8vdhQakGjFIvNrHA6y3HCF1CInLy+QLq8dTJPQ1b+KDUqDFctkdRW44e1w==", + "dev": true, + "dependencies": { + "emoji-regex": "^7.0.1", + "is-fullwidth-code-point": "^2.0.0", + "strip-ansi": "^5.1.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/strip-ansi": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-5.2.0.tgz", + "integrity": "sha512-DuRs1gKbBqsMKIZlrffwlug8MHkcnpjs5VPmL1PAh+mA30U0DTotfDZ0d2UUsXpPmPmMMJ6W773MaA3J+lbiWA==", + "dev": true, + "dependencies": { + "ansi-regex": "^4.1.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/uuid": { + "version": "3.4.0", + "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.4.0.tgz", + "integrity": "sha512-HjSDRw6gZE5JMggctHBcjVak08+KEVhSIiDzFnT9S9aegmp85S/bReBVTb4QTFaRNptJ9kuYaNhnbNEOkbKb/A==", + "deprecated": "Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.", + "dev": true, + "bin": { + "uuid": "bin/uuid" + } + }, + "node_modules/nyc/node_modules/wrap-ansi": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-5.1.0.tgz", + "integrity": "sha512-QC1/iN/2/RPVJ5jYK8BGttj5z83LmSKmvbvrXPNCLZSEb32KKVDJDl/MOt2N01qU2H/FkzEa9PKto1BqDjtd7Q==", + "dev": true, + "dependencies": { + "ansi-styles": "^3.2.0", + "string-width": "^3.0.0", + "strip-ansi": "^5.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/nyc/node_modules/y18n": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/y18n/-/y18n-4.0.3.tgz", + "integrity": "sha512-JKhqTOwSrqNA1NY5lSztJ1GrBiUodLMmIZuLiDaMRJ+itFd+ABVE8XBjOvIWL+rSqNDC74LCSFmlb/U4UZ4hJQ==", + "dev": true + }, + "node_modules/nyc/node_modules/yargs": { + "version": "13.3.2", + "resolved": "https://registry.npmjs.org/yargs/-/yargs-13.3.2.tgz", + "integrity": "sha512-AX3Zw5iPruN5ie6xGRIDgqkT+ZhnRlZMLMHAs8tg7nRruy2Nb+i5o9bwghAogtM08q1dpr2LVoS8KSTMYpWXUw==", + "dev": true, + "dependencies": { + "cliui": "^5.0.0", + "find-up": "^3.0.0", + "get-caller-file": "^2.0.1", + "require-directory": "^2.1.1", + "require-main-filename": "^2.0.0", + "set-blocking": "^2.0.0", + "string-width": "^3.0.0", + "which-module": "^2.0.0", + "y18n": "^4.0.0", + "yargs-parser": "^13.1.2" + } + }, + "node_modules/nyc/node_modules/yargs-parser": { + "version": "13.1.2", + "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-13.1.2.tgz", + "integrity": "sha512-3lbsNRf/j+A4QuSZfDRA7HRSfWrzO0YjqTJd5kjAq37Zep1CEgaYmrH9Q3GwPiB9cHyd1Y1UwggGhJGoxipbzg==", + "dev": true, + "dependencies": { + "camelcase": "^5.0.0", + "decamelize": "^1.2.0" + } + }, + "node_modules/oauth2orize": { + "version": "1.11.1", + "resolved": "https://registry.npmjs.org/oauth2orize/-/oauth2orize-1.11.1.tgz", + "integrity": "sha512-9dSx/Gwm0J2Rvj4RH9+h7iXVnRXZ6biwWRgb2dCeQhCosODS0nYdM9I/G7BUGsjbgn0pHjGcn1zcCRtzj2SlRA==", + "dependencies": { + "debug": "2.x.x", + "uid2": "0.0.x", + "utils-merge": "1.x.x" + }, + "engines": { + "node": ">= 0.4.0" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/jaredhanson" + } + }, + "node_modules/object-assign": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz", + "integrity": "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/object-get": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/object-get/-/object-get-2.1.1.tgz", + "integrity": "sha512-7n4IpLMzGGcLEMiQKsNR7vCe+N5E9LORFrtNUVy4sO3dj9a3HedZCxEL2T7QuLhcHN1NBuBsMOKaOsAYI9IIvg==", + "dev": true + }, + "node_modules/object-inspect": { + "version": "1.12.3", + "resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.12.3.tgz", + "integrity": "sha512-geUvdk7c+eizMNUDkRpW1wJwgfOiOeHbxBR/hLXK1aT6zmVSO0jsQcs7fj6MGw89jC/cjGfLcNOrtMYtGqm81g==", + "dev": true, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/object-to-spawn-args": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/object-to-spawn-args/-/object-to-spawn-args-2.0.1.tgz", + "integrity": "sha512-6FuKFQ39cOID+BMZ3QaphcC8Y4cw6LXBLyIgPU+OhIYwviJamPAn+4mITapnSBQrejB+NNp+FMskhD8Cq+Ys3w==", + "dev": true, + "engines": { + "node": ">=8.0.0" + } + }, + "node_modules/object.defaults": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/object.defaults/-/object.defaults-1.1.0.tgz", + "integrity": "sha512-c/K0mw/F11k4dEUBMW8naXUuBuhxRCfG7W+yFy8EcijU/rSmazOUd1XAEEe6bC0OuXY4HUKjTJv7xbxIMqdxrA==", + "dev": true, + "dependencies": { + "array-each": "^1.0.1", + "array-slice": "^1.0.0", + "for-own": "^1.0.0", + "isobject": "^3.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/object.map": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/object.map/-/object.map-1.0.1.tgz", + "integrity": "sha512-3+mAJu2PLfnSVGHwIWubpOFLscJANBKuB/6A4CxBstc4aqwQY0FWcsppuy4jU5GSB95yES5JHSI+33AWuS4k6w==", + "dev": true, + "dependencies": { + "for-own": "^1.0.0", + "make-iterator": "^1.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/object.pick": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/object.pick/-/object.pick-1.3.0.tgz", + "integrity": "sha512-tqa/UMy/CCoYmj+H5qc07qvSL9dqcs/WZENZ1JbtWBlATP+iVOe778gE6MSijnyCnORzDuX6hU+LA4SZ09YjFQ==", + "dev": true, + "dependencies": { + "isobject": "^3.0.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/on-finished": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.3.0.tgz", + "integrity": "sha512-ikqdkGAAyf/X/gPhXGvfgAytDZtDbr+bkNUJ0N9h5MI/dmdgCs3l6hoHrcUv41sRKew3jIwrp4qQDXiK99Utww==", + "dependencies": { + "ee-first": "1.1.1" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/on-headers": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/on-headers/-/on-headers-1.0.2.tgz", + "integrity": "sha512-pZAE+FJLoyITytdqK0U5s+FIpjN0JP3OzFi/u8Rx+EV5/W+JTWGXG8xFzevE7AjBfDqHv/8vL8qQsIhHnqRkrA==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/once": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz", + "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==", + "dependencies": { + "wrappy": "1" + } + }, + "node_modules/opted": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/opted/-/opted-1.0.2.tgz", + "integrity": "sha512-uEvunmdmKcSFiBSmnY2E9E/HbghO5yc1J0yNmq7T18YkAJeWNlo33e6VYKkRK4eudVrpvvlLdemAeAuL6rZxjQ==", + "dev": true, + "dependencies": { + "lodash": "^4.17.4" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/os-homedir": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/os-homedir/-/os-homedir-1.0.2.tgz", + "integrity": "sha512-B5JU3cabzk8c67mRRd3ECmROafjYMXbuzlwtqdM8IbS8ktlTix8aFGb2bAGKrSRIlnfKwovGUUr72JUPyOb6kQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/os-tmpdir": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/os-tmpdir/-/os-tmpdir-1.0.2.tgz", + "integrity": "sha512-D2FR03Vir7FIu45XBY20mTb+/ZSWB00sjU9jdQXt83gDrI4Ztz5Fs7/yy74g2N5SVQY4xY1qDr4rNddwYRVX0g==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/osenv": { + "version": "0.1.5", + "resolved": "https://registry.npmjs.org/osenv/-/osenv-0.1.5.tgz", + "integrity": "sha512-0CWcCECdMVc2Rw3U5w9ZjqX6ga6ubk1xDVKxtBQPK7wis/0F2r9T6k4ydGYhecl7YUBxBVxhL5oisPsNxAPe2g==", + "dev": true, + "dependencies": { + "os-homedir": "^1.0.0", + "os-tmpdir": "^1.0.0" + } + }, + "node_modules/p-cancelable": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/p-cancelable/-/p-cancelable-2.1.1.tgz", + "integrity": "sha512-BZOr3nRQHOntUjTrH8+Lh54smKHoHyur8We1V8DSMVrl5A2malOOwuJRnKRDjSnkoeBh4at6BwEnb5I7Jl31wg==", + "engines": { + "node": ">=8" + } + }, + "node_modules/p-limit": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/p-limit/-/p-limit-3.1.0.tgz", + "integrity": "sha512-TYOanM3wGwNGsZN2cVTYPArw454xnXj5qmWF1bEoAc4+cU/ol7GVh7odevjp1FNHduHc3KZMcFduxU5Xc6uJRQ==", + "dev": true, + "dependencies": { + "yocto-queue": "^0.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/p-locate": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/p-locate/-/p-locate-5.0.0.tgz", + "integrity": "sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==", + "dev": true, + "dependencies": { + "p-limit": "^3.0.2" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/p-try": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/p-try/-/p-try-2.2.0.tgz", + "integrity": "sha512-R4nPAVTAU0B9D35/Gk3uJf/7XYbQcyohSKdvAxIRSNghFl4e71hVoGnBNQz9cWaXxO2I10KTC+3jMdvvoKw6dQ==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/package-hash": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/package-hash/-/package-hash-3.0.0.tgz", + "integrity": "sha512-lOtmukMDVvtkL84rJHI7dpTYq+0rli8N2wlnqUcBuDWCfVhRUfOmnR9SsoHFMLpACvEV60dX7rd0rFaYDZI+FA==", + "dev": true, + "dependencies": { + "graceful-fs": "^4.1.15", + "hasha": "^3.0.0", + "lodash.flattendeep": "^4.4.0", + "release-zalgo": "^1.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/package-json": { + "version": "6.5.0", + "resolved": "https://registry.npmjs.org/package-json/-/package-json-6.5.0.tgz", + "integrity": "sha512-k3bdm2n25tkyxcjSKzB5x8kfVxlMdgsbPr0GkZcwHsLpba6cBjqCt1KlcChKEvxHIcTB1FVMuwoijZ26xex5MQ==", + "dev": true, + "dependencies": { + "got": "^9.6.0", + "registry-auth-token": "^4.0.0", + "registry-url": "^5.0.0", + "semver": "^6.2.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/package-json/node_modules/@sindresorhus/is": { + "version": "0.14.0", + "resolved": "https://registry.npmjs.org/@sindresorhus/is/-/is-0.14.0.tgz", + "integrity": "sha512-9NET910DNaIPngYnLLPeg+Ogzqsi9uM4mSboU5y6p8S5DzMTVEsJZrawi+BoDNUVBa2DhJqQYUFvMDfgU062LQ==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/package-json/node_modules/@szmarczak/http-timer": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@szmarczak/http-timer/-/http-timer-1.1.2.tgz", + "integrity": "sha512-XIB2XbzHTN6ieIjfIMV9hlVcfPU26s2vafYWQcZHWXHOxiaRZYEDKEwdl129Zyg50+foYV2jCgtrqSA6qNuNSA==", + "dev": true, + "dependencies": { + "defer-to-connect": "^1.0.1" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/package-json/node_modules/cacheable-request": { + "version": "6.1.0", + "resolved": "https://registry.npmjs.org/cacheable-request/-/cacheable-request-6.1.0.tgz", + "integrity": "sha512-Oj3cAGPCqOZX7Rz64Uny2GYAZNliQSqfbePrgAQ1wKAihYmCUnraBtJtKcGR4xz7wF+LoJC+ssFZvv5BgF9Igg==", + "dev": true, + "dependencies": { + "clone-response": "^1.0.2", + "get-stream": "^5.1.0", + "http-cache-semantics": "^4.0.0", + "keyv": "^3.0.0", + "lowercase-keys": "^2.0.0", + "normalize-url": "^4.1.0", + "responselike": "^1.0.2" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/package-json/node_modules/decompress-response": { + "version": "3.3.0", + "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-3.3.0.tgz", + "integrity": "sha512-BzRPQuY1ip+qDonAOz42gRm/pg9F768C+npV/4JOsxRC2sq+Rlk+Q4ZCAsOhnIaMrgarILY+RMUIvMmmX1qAEA==", + "dev": true, + "dependencies": { + "mimic-response": "^1.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/package-json/node_modules/defer-to-connect": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/defer-to-connect/-/defer-to-connect-1.1.3.tgz", + "integrity": "sha512-0ISdNousHvZT2EiFlZeZAHBUvSxmKswVCEf8hW7KWgG4a8MVEu/3Vb6uWYozkjylyCxe0JBIiRB1jV45S70WVQ==", + "dev": true + }, + "node_modules/package-json/node_modules/got": { + "version": "9.6.0", + "resolved": "https://registry.npmjs.org/got/-/got-9.6.0.tgz", + "integrity": "sha512-R7eWptXuGYxwijs0eV+v3o6+XH1IqVK8dJOEecQfTmkncw9AV4dcw/Dhxi8MdlqPthxxpZyizMzyg8RTmEsG+Q==", + "dev": true, + "dependencies": { + "@sindresorhus/is": "^0.14.0", + "@szmarczak/http-timer": "^1.1.2", + "cacheable-request": "^6.0.0", + "decompress-response": "^3.3.0", + "duplexer3": "^0.1.4", + "get-stream": "^4.1.0", + "lowercase-keys": "^1.0.1", + "mimic-response": "^1.0.1", + "p-cancelable": "^1.0.0", + "to-readable-stream": "^1.0.0", + "url-parse-lax": "^3.0.0" + }, + "engines": { + "node": ">=8.6" + } + }, + "node_modules/package-json/node_modules/got/node_modules/get-stream": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-4.1.0.tgz", + "integrity": "sha512-GMat4EJ5161kIy2HevLlr4luNjBgvmj413KaQA7jt4V8B4RDsfpHk7WQ9GVqfYyyx8OS/L66Kox+rJRNklLK7w==", + "dev": true, + "dependencies": { + "pump": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/package-json/node_modules/got/node_modules/lowercase-keys": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/lowercase-keys/-/lowercase-keys-1.0.1.tgz", + "integrity": "sha512-G2Lj61tXDnVFFOi8VZds+SoQjtQC3dgokKdDG2mTm1tx4m50NUHBOZSBwQQHyy0V12A0JTG4icfZQH+xPyh8VA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/package-json/node_modules/json-buffer": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/json-buffer/-/json-buffer-3.0.0.tgz", + "integrity": "sha512-CuUqjv0FUZIdXkHPI8MezCnFCdaTAacej1TZYulLoAg1h/PhwkdXFN4V/gzY4g+fMBCOV2xF+rp7t2XD2ns/NQ==", + "dev": true + }, + "node_modules/package-json/node_modules/keyv": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/keyv/-/keyv-3.1.0.tgz", + "integrity": "sha512-9ykJ/46SN/9KPM/sichzQ7OvXyGDYKGTaDlKMGCAlg2UK8KRy4jb0d8sFc+0Tt0YYnThq8X2RZgCg74RPxgcVA==", + "dev": true, + "dependencies": { + "json-buffer": "3.0.0" + } + }, + "node_modules/package-json/node_modules/normalize-url": { + "version": "4.5.1", + "resolved": "https://registry.npmjs.org/normalize-url/-/normalize-url-4.5.1.tgz", + "integrity": "sha512-9UZCFRHQdNrfTpGg8+1INIg93B6zE0aXMVFkw1WFwvO4SlZywU6aLg5Of0Ap/PgcbSw4LNxvMWXMeugwMCX0AA==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/package-json/node_modules/p-cancelable": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/p-cancelable/-/p-cancelable-1.1.0.tgz", + "integrity": "sha512-s73XxOZ4zpt1edZYZzvhqFa6uvQc1vwUa0K0BdtIZgQMAJj9IbebH+JkgKZc9h+B05PKHLOTl4ajG1BmNrVZlw==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/package-json/node_modules/responselike": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/responselike/-/responselike-1.0.2.tgz", + "integrity": "sha512-/Fpe5guzJk1gPqdJLJR5u7eG/gNY4nImjbRDaVWVMRhne55TCmj2i9Q+54PBRfatRC8v/rIiv9BN0pMd9OV5EQ==", + "dev": true, + "dependencies": { + "lowercase-keys": "^1.0.0" + } + }, + "node_modules/package-json/node_modules/responselike/node_modules/lowercase-keys": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/lowercase-keys/-/lowercase-keys-1.0.1.tgz", + "integrity": "sha512-G2Lj61tXDnVFFOi8VZds+SoQjtQC3dgokKdDG2mTm1tx4m50NUHBOZSBwQQHyy0V12A0JTG4icfZQH+xPyh8VA==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/package-json/node_modules/semver": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/semver/-/semver-6.3.0.tgz", + "integrity": "sha512-b39TBaTSfV6yBrapU89p5fKekE2m/NwnDocOVruQFS1/veMgdzuPcnOM34M6CwxW8jH/lxEa5rBoDeUwu5HHTw==", + "dev": true, + "bin": { + "semver": "bin/semver.js" + } + }, + "node_modules/pad-stream": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/pad-stream/-/pad-stream-2.0.0.tgz", + "integrity": "sha512-3QeQw19K48BQzUGZ9dEf/slX5Jbfy5ZeBTma2XICketO7kFNK7omF00riVcecOKN+DSiJZcK2em1eYKaVOeXKg==", + "dev": true, + "dependencies": { + "pumpify": "^1.3.3", + "split2": "^2.1.1", + "through2": "^2.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/pad-stream/node_modules/split2": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/split2/-/split2-2.2.0.tgz", + "integrity": "sha512-RAb22TG39LhI31MbreBgIuKiIKhVsawfTgEGqKHTK87aG+ul/PB8Sqoi3I7kVdRWiCfrKxK3uo4/YUkpNvhPbw==", + "dev": true, + "dependencies": { + "through2": "^2.0.2" + } + }, + "node_modules/parse-filepath": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/parse-filepath/-/parse-filepath-1.0.2.tgz", + "integrity": "sha512-FwdRXKCohSVeXqwtYonZTXtbGJKrn+HNyWDYVcp5yuJlesTwNH4rsmRZ+GrKAPJ5bLpRxESMeS+Rl0VCHRvB2Q==", + "dev": true, + "dependencies": { + "is-absolute": "^1.0.0", + "map-cache": "^0.2.0", + "path-root": "^0.1.1" + }, + "engines": { + "node": ">=0.8" + } + }, + "node_modules/parse-json": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/parse-json/-/parse-json-5.2.0.tgz", + "integrity": "sha512-ayCKvm/phCGxOkYRSCM82iDwct8/EonSEgCSxWxD7ve6jHggsFl4fZVQBPRNgQoKiuV/odhFrGzQXZwbifC8Rg==", + "dev": true, + "dependencies": { + "@babel/code-frame": "^7.0.0", + "error-ex": "^1.3.1", + "json-parse-even-better-errors": "^2.3.0", + "lines-and-columns": "^1.1.6" + }, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/parse-passwd": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/parse-passwd/-/parse-passwd-1.0.0.tgz", + "integrity": "sha512-1Y1A//QUXEZK7YKz+rD9WydcE1+EuPr6ZBgKecAB8tmoW6UFv0NREVJe1p+jRxtThkcbbKkfwIbWJe/IeE6m2Q==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/parse5": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/parse5/-/parse5-6.0.1.tgz", + "integrity": "sha512-Ofn/CTFzRGTTxwpNEs9PP93gXShHcTq255nzRYSKe8AkVpZY7e1fpmTfOyoIvjP5HG7Z2ZM7VS9PPhQGW2pOpw==" + }, + "node_modules/parse5-htmlparser2-tree-adapter": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/parse5-htmlparser2-tree-adapter/-/parse5-htmlparser2-tree-adapter-6.0.1.tgz", + "integrity": "sha512-qPuWvbLgvDGilKc5BoicRovlT4MtYT6JfJyBOMDsKoiT+GiuP5qyrPCnR9HcPECIJJmZh5jRndyNThnhhb/vlA==", + "dependencies": { + "parse5": "^6.0.1" + } + }, + "node_modules/parseurl": { + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz", + "integrity": "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/passport": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/passport/-/passport-0.5.2.tgz", + "integrity": "sha512-w9n/Ot5I7orGD4y+7V3EFJCQEznE5RxHamUxcqLT2QoJY0f2JdN8GyHonYFvN0Vz+L6lUJfVhrk2aZz2LbuREw==", + "dependencies": { + "passport-strategy": "1.x.x", + "pause": "0.0.1" + }, + "engines": { + "node": ">= 0.4.0" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/jaredhanson" + } + }, + "node_modules/passport-http-bearer": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/passport-http-bearer/-/passport-http-bearer-1.0.1.tgz", + "integrity": "sha512-SELQM+dOTuMigr9yu8Wo4Fm3ciFfkMq5h/ZQ8ffi4ELgZrX1xh9PlglqZdcUZ1upzJD/whVyt+YWF62s3U6Ipw==", + "dependencies": { + "passport-strategy": "1.x.x" + }, + "engines": { + "node": ">= 0.4.0" + } + }, + "node_modules/passport-oauth2-client-password": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/passport-oauth2-client-password/-/passport-oauth2-client-password-0.1.2.tgz", + "integrity": "sha512-GHQH4UtaEZvCLulAxGKHYoSsPRoPRmGsdmaZtMh5nmz80yMLQbdMA9Bg2sp4/UW3PIxJH/143hVjPTiXaNngTQ==", + "dependencies": { + "passport-strategy": "1.x.x" + }, + "engines": { + "node": ">= 0.4.0" + } + }, + "node_modules/passport-strategy": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/passport-strategy/-/passport-strategy-1.0.0.tgz", + "integrity": "sha512-CB97UUvDKJde2V0KDWWB3lyf6PC3FaZP7YxZ2G8OAtn9p4HI9j9JLP9qjOGZFvyl8uwNT8qM+hGnz/n16NI7oA==", + "engines": { + "node": ">= 0.4.0" + } + }, + "node_modules/path-exists": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz", + "integrity": "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/path-is-absolute": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/path-is-absolute/-/path-is-absolute-1.0.1.tgz", + "integrity": "sha512-AVbw3UJ2e9bq64vSaS9Am0fje1Pa8pbGqTTsmXfaIiMpnr5DlDhfJOuLj9Sf95ZPVDAUerDfEk88MPmPe7UCQg==", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/path-key": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz", + "integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/path-parse": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz", + "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==", + "dev": true + }, + "node_modules/path-root": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/path-root/-/path-root-0.1.1.tgz", + "integrity": "sha512-QLcPegTHF11axjfojBIoDygmS2E3Lf+8+jI6wOVmNVenrKSo3mFdSGiIgdSHenczw3wPtlVMQaFVwGmM7BJdtg==", + "dev": true, + "dependencies": { + "path-root-regex": "^0.1.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/path-root-regex": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/path-root-regex/-/path-root-regex-0.1.2.tgz", + "integrity": "sha512-4GlJ6rZDhQZFE0DPVKh0e9jmZ5egZfxTkp7bcRDuPlJXbAwhxcl2dINPUAsjLdejqaLsCeg8axcLjIbvBjN4pQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/path-to-regexp": { + "version": "0.1.7", + "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz", + "integrity": "sha512-5DFkuoqlv1uYQKxy8omFBeJPQcdoE07Kv2sferDCrAq1ohOU+MSDswDIbnx3YAM60qIOnYa53wBhXW0EbMonrQ==" + }, + "node_modules/path-type": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/path-type/-/path-type-3.0.0.tgz", + "integrity": "sha512-T2ZUsdZFHgA3u4e5PfPbjd7HDDpxPnQb5jN0SrDsjNSuVXHJqtwTnWqG0B1jZrgmJ/7lj1EmVIByWt1gxGkWvg==", + "dev": true, + "dependencies": { + "pify": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/path-type/node_modules/pify": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/pify/-/pify-3.0.0.tgz", + "integrity": "sha512-C3FsVNH1udSEX48gGX1xfvwTWfsYWj5U+8/uK15BGzIGrKoUpghX8hWZwa/OFnakBiiVNmBvemTJR5mcy7iPcg==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/pause": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/pause/-/pause-0.0.1.tgz", + "integrity": "sha512-KG8UEiEVkR3wGEb4m5yZkVCzigAD+cVEJck2CzYZO37ZGJfctvVptVO192MwrtPhzONn6go8ylnOdMhKqi4nfg==" + }, + "node_modules/picomatch": { + "version": "2.3.1", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz", + "integrity": "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==", + "dev": true, + "engines": { + "node": ">=8.6" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, + "node_modules/pify": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/pify/-/pify-4.0.1.tgz", + "integrity": "sha512-uB80kBFb/tfd68bVleG9T5GGsGPjJrLAUpR5PZIrhBnIaRTQRjqdJSsIKkOP6OAIFbj7GOrcudc5pNjZ+geV2g==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/pkg-dir": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/pkg-dir/-/pkg-dir-3.0.0.tgz", + "integrity": "sha512-/E57AYkoeQ25qkxMj5PBOVgF8Kiu/h7cYS30Z5+R7WaiCCBfLq58ZI/dSeaEKb9WVJV5n/03QwrN3IeWIFllvw==", + "dev": true, + "dependencies": { + "find-up": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/pkg-dir/node_modules/find-up": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/find-up/-/find-up-3.0.0.tgz", + "integrity": "sha512-1yD6RmLI1XBfxugvORwlck6f75tYL+iR0jqwsOrOxMZyGYqUuDhJ0l4AXdO1iX/FTs9cBAMEk1gWSEx1kSbylg==", + "dev": true, + "dependencies": { + "locate-path": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/pkg-dir/node_modules/locate-path": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/locate-path/-/locate-path-3.0.0.tgz", + "integrity": "sha512-7AO748wWnIhNqAuaty2ZWHkQHRSNfPVIsPIfwEOWO22AmaoVrWavlOcMR5nzTLNYvp36X220/maaRsrec1G65A==", + "dev": true, + "dependencies": { + "p-locate": "^3.0.0", + "path-exists": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/pkg-dir/node_modules/p-limit": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/p-limit/-/p-limit-2.3.0.tgz", + "integrity": "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w==", + "dev": true, + "dependencies": { + "p-try": "^2.0.0" + }, + "engines": { + "node": ">=6" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/pkg-dir/node_modules/p-locate": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/p-locate/-/p-locate-3.0.0.tgz", + "integrity": "sha512-x+12w/To+4GFfgJhBEpiDcLozRJGegY+Ei7/z0tSLkMmxGZNybVMSfWj9aJn8Z5Fc7dBUNJOOVgPv2H7IwulSQ==", + "dev": true, + "dependencies": { + "p-limit": "^2.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/pkg-dir/node_modules/path-exists": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-3.0.0.tgz", + "integrity": "sha512-bpC7GYwiDYQ4wYLe+FA8lhRjhQCMcQGuSgGGqDkg/QerRWw9CmGRT0iSOVRSZJ29NMLZgIzqaljJ63oaL4NIJQ==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/prepend-http": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/prepend-http/-/prepend-http-2.0.0.tgz", + "integrity": "sha512-ravE6m9Atw9Z/jjttRUZ+clIXogdghyZAuWJ3qEzjT+jI/dL1ifAqhZeC5VHzQp1MSt1+jxKkFNemj/iO7tVUA==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/pretty-bytes": { + "version": "5.6.0", + "resolved": "https://registry.npmjs.org/pretty-bytes/-/pretty-bytes-5.6.0.tgz", + "integrity": "sha512-FFw039TmrBqFK8ma/7OL3sDz/VytdtJr044/QUJtH0wK9lb9jLq9tJyIxUwtQJHwar2BqtiA4iCWSwo9JLkzFg==", + "dev": true, + "engines": { + "node": ">=6" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/process-nextick-args": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.1.tgz", + "integrity": "sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==" + }, + "node_modules/proxy": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/proxy/-/proxy-1.0.2.tgz", + "integrity": "sha512-KNac2ueWRpjbUh77OAFPZuNdfEqNynm9DD4xHT14CccGpW8wKZwEkN0yjlb7X9G9Z9F55N0Q+1z+WfgAhwYdzQ==", + "dev": true, + "dependencies": { + "args": "5.0.1", + "basic-auth-parser": "0.0.2", + "debug": "^4.1.1" + }, + "bin": { + "proxy": "bin/proxy.js" + } + }, + "node_modules/proxy-addr": { + "version": "2.0.7", + "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz", + "integrity": "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==", + "dependencies": { + "forwarded": "0.2.0", + "ipaddr.js": "1.9.1" + }, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/proxy/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dev": true, + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/proxy/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==", + "dev": true + }, + "node_modules/pseudomap": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/pseudomap/-/pseudomap-1.0.2.tgz", + "integrity": "sha512-b/YwNhb8lk1Zz2+bXXpS/LK9OisiZZ1SNsSLxN1x2OXVEhW2Ckr/7mWE5vrC1ZTiJlD9g19jWszTmJsB+oEpFQ==" + }, + "node_modules/psl": { + "version": "1.9.0", + "resolved": "https://registry.npmjs.org/psl/-/psl-1.9.0.tgz", + "integrity": "sha512-E/ZsdU4HLs/68gYzgGTkMicWTLPdAftJLfJFlLUAAKZGkStNU72sZjT66SnMDVOfOWY/YAoiD7Jxa9iHvngcag==" + }, + "node_modules/pstree.remy": { + "version": "1.1.8", + "resolved": "https://registry.npmjs.org/pstree.remy/-/pstree.remy-1.1.8.tgz", + "integrity": "sha512-77DZwxQmxKnu3aR542U+X8FypNzbfJ+C5XQDk3uWjWxn6151aIMGthWYRXTqT1E5oJvg+ljaa2OJi+VfvCOQ8w==", + "dev": true + }, + "node_modules/pump": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.0.tgz", + "integrity": "sha512-LwZy+p3SFs1Pytd/jYct4wpv49HiYCqd9Rlc5ZVdk0V+8Yzv6jR5Blk3TRmPL1ft69TxP0IMZGJ+WPFU2BFhww==", + "dependencies": { + "end-of-stream": "^1.1.0", + "once": "^1.3.1" + } + }, + "node_modules/pumpify": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/pumpify/-/pumpify-1.5.1.tgz", + "integrity": "sha512-oClZI37HvuUJJxSKKrC17bZ9Cu0ZYhEAGPsPUy9KlMUmv9dKX2o77RUmq7f3XjIxbwyGwYzbzQ1L2Ks8sIradQ==", + "dev": true, + "dependencies": { + "duplexify": "^3.6.0", + "inherits": "^2.0.3", + "pump": "^2.0.0" + } + }, + "node_modules/pumpify/node_modules/duplexify": { + "version": "3.7.1", + "resolved": "https://registry.npmjs.org/duplexify/-/duplexify-3.7.1.tgz", + "integrity": "sha512-07z8uv2wMyS51kKhD1KsdXJg5WQ6t93RneqRxUHnskXVtlYYkLqM0gqStQZ3pj073g687jPCHrqNfCzawLYh5g==", + "dev": true, + "dependencies": { + "end-of-stream": "^1.0.0", + "inherits": "^2.0.1", + "readable-stream": "^2.0.0", + "stream-shift": "^1.0.0" + } + }, + "node_modules/pumpify/node_modules/pump": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/pump/-/pump-2.0.1.tgz", + "integrity": "sha512-ruPMNRkN3MHP1cWJc9OWr+T/xDP0jhXYCLfJcBuX54hhfIBnaQmAUMfDcG4DM5UMWByBbJY69QSphm3jtDKIkA==", + "dev": true, + "dependencies": { + "end-of-stream": "^1.1.0", + "once": "^1.3.1" + } + }, + "node_modules/pumpify/node_modules/readable-stream": { + "version": "2.3.8", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.8.tgz", + "integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==", + "dev": true, + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.3", + "isarray": "~1.0.0", + "process-nextick-args": "~2.0.0", + "safe-buffer": "~5.1.1", + "string_decoder": "~1.1.1", + "util-deprecate": "~1.0.1" + } + }, + "node_modules/pumpify/node_modules/string_decoder": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz", + "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==", + "dev": true, + "dependencies": { + "safe-buffer": "~5.1.0" + } + }, + "node_modules/punycode": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.0.tgz", + "integrity": "sha512-rRV+zQD8tVFys26lAGR9WUuS4iUAngJScM+ZRSKtvl5tKeZ2t5bvdNFdNHBW9FWR4guGHlgmsZ1G7BSm2wTbuA==", + "engines": { + "node": ">=6" + } + }, + "node_modules/pupa": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/pupa/-/pupa-2.1.1.tgz", + "integrity": "sha512-l1jNAspIBSFqbT+y+5FosojNpVpF94nlI+wDUpqP9enwOTfHx9f0gh5nB96vl+6yTpsJsypeNrwfzPrKuHB41A==", + "dev": true, + "dependencies": { + "escape-goat": "^2.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/qs": { + "version": "6.9.6", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.9.6.tgz", + "integrity": "sha512-TIRk4aqYLNoJUbd+g2lEdz5kLWIuTMRagAXxl78Q0RiVjAOugHmeKNGdd3cwo/ktpf9aL9epCfFqWDEKysUlLQ==", + "engines": { + "node": ">=0.6" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/quick-lru": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/quick-lru/-/quick-lru-5.1.1.tgz", + "integrity": "sha512-WuyALRjWPDGtt/wzJiadO5AXY+8hZ80hVpe6MyivgraREW751X3SbhRvG3eLKOYN+8VEvqLcf3wdnt44Z4S4SA==", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/random-bytes": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/random-bytes/-/random-bytes-1.0.0.tgz", + "integrity": "sha512-iv7LhNVO047HzYR3InF6pUcUsPQiHTM1Qal51DcGSuZFBil1aBBWG5eHPNek7bvILMaYJ/8RU1e8w1AMdHmLQQ==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/randombytes": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/randombytes/-/randombytes-2.1.0.tgz", + "integrity": "sha512-vYl3iOX+4CKUWuxGi9Ukhie6fsqXqS9FE2Zaic4tNFD2N2QQaXOMFbuKK4QmDHC0JO6B1Zp41J0LpT0oR68amQ==", + "dev": true, + "dependencies": { + "safe-buffer": "^5.1.0" + } + }, + "node_modules/range-parser": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz", + "integrity": "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/raw-body": { + "version": "2.4.3", + "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.4.3.tgz", + "integrity": "sha512-UlTNLIcu0uzb4D2f4WltY6cVjLi+/jEN4lgEUj3E04tpMDpUlkBo/eSn6zou9hum2VMNpCCUone0O0WeJim07g==", + "dependencies": { + "bytes": "3.1.2", + "http-errors": "1.8.1", + "iconv-lite": "0.4.24", + "unpipe": "1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/raw-body/node_modules/bytes": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz", + "integrity": "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/raw-body/node_modules/iconv-lite": { + "version": "0.4.24", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz", + "integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==", + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/rc": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz", + "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==", + "dev": true, + "dependencies": { + "deep-extend": "^0.6.0", + "ini": "~1.3.0", + "minimist": "^1.2.0", + "strip-json-comments": "~2.0.1" + }, + "bin": { + "rc": "cli.js" + } + }, + "node_modules/rc/node_modules/ini": { + "version": "1.3.8", + "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz", + "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==", + "dev": true + }, + "node_modules/rc/node_modules/strip-json-comments": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz", + "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/read": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/read/-/read-1.0.7.tgz", + "integrity": "sha512-rSOKNYUmaxy0om1BNjMN4ezNT6VKK+2xF4GBhc81mkH7L60i6dp8qPYrkndNLT3QPphoII3maL9PVC9XmhHwVQ==", + "dependencies": { + "mute-stream": "~0.0.4" + }, + "engines": { + "node": ">=0.8" + } + }, + "node_modules/read-pkg": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/read-pkg/-/read-pkg-5.2.0.tgz", + "integrity": "sha512-Ug69mNOpfvKDAc2Q8DRpMjjzdtrnv9HcSMX+4VsZxD1aZ6ZzrIE7rlzXBtWTyhULSMKg076AW6WR5iZpD0JiOg==", + "dev": true, + "dependencies": { + "@types/normalize-package-data": "^2.4.0", + "normalize-package-data": "^2.5.0", + "parse-json": "^5.0.0", + "type-fest": "^0.6.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/read-pkg-up": { + "version": "7.0.1", + "resolved": "https://registry.npmjs.org/read-pkg-up/-/read-pkg-up-7.0.1.tgz", + "integrity": "sha512-zK0TB7Xd6JpCLmlLmufqykGE+/TlOePD6qKClNW7hHDKFh/J7/7gCWGR7joEQEW1bKq3a3yUZSObOoWLFQ4ohg==", + "dev": true, + "dependencies": { + "find-up": "^4.1.0", + "read-pkg": "^5.2.0", + "type-fest": "^0.8.1" + }, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/read-pkg-up/node_modules/find-up": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz", + "integrity": "sha512-PpOwAdQ/YlXQ2vj8a3h8IipDuYRi3wceVQQGYWxNINccq40Anw7BlsEXCMbt1Zt+OLA6Fq9suIpIWD0OsnISlw==", + "dev": true, + "dependencies": { + "locate-path": "^5.0.0", + "path-exists": "^4.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/read-pkg-up/node_modules/locate-path": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz", + "integrity": "sha512-t7hw9pI+WvuwNJXwk5zVHpyhIqzg2qTlklJOf0mVxGSbe3Fp2VieZcduNYjaLDoy6p9uGpQEGWG87WpMKlNq8g==", + "dev": true, + "dependencies": { + "p-locate": "^4.1.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/read-pkg-up/node_modules/p-limit": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/p-limit/-/p-limit-2.3.0.tgz", + "integrity": "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w==", + "dev": true, + "dependencies": { + "p-try": "^2.0.0" + }, + "engines": { + "node": ">=6" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/read-pkg-up/node_modules/p-locate": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/p-locate/-/p-locate-4.1.0.tgz", + "integrity": "sha512-R79ZZ/0wAxKGu3oYMlz8jy/kbhsNrS7SKZ7PxEHBgJ5+F2mtFW2fK2cOtBh1cHYkQsbzFV7I+EoRKe6Yt0oK7A==", + "dev": true, + "dependencies": { + "p-limit": "^2.2.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/read-pkg/node_modules/type-fest": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.6.0.tgz", + "integrity": "sha512-q+MB8nYR1KDLrgr4G5yemftpMC7/QLqVndBmEEdqzmNj5dcFOO4Oo8qlwZE3ULT3+Zim1F8Kq4cBnikNhlCMlg==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/readable-stream": { + "version": "3.6.2", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz", + "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==", + "dependencies": { + "inherits": "^2.0.3", + "string_decoder": "^1.1.1", + "util-deprecate": "^1.0.1" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/readdir-glob": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/readdir-glob/-/readdir-glob-1.1.2.tgz", + "integrity": "sha512-6RLVvwJtVwEDfPdn6X6Ille4/lxGl0ATOY4FN/B9nxQcgOazvvI0nodiD19ScKq0PvA/29VpaOQML36o5IzZWA==", + "dev": true, + "dependencies": { + "minimatch": "^5.1.0" + } + }, + "node_modules/readdir-glob/node_modules/brace-expansion": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.1.tgz", + "integrity": "sha512-XnAIvQ8eM+kC6aULx6wuQiwVsnzsi9d3WxzV3FpWTGA19F621kwdbsAcFKXgKUHZWsy+mY6iL1sHTxWEFCytDA==", + "dev": true, + "dependencies": { + "balanced-match": "^1.0.0" + } + }, + "node_modules/readdir-glob/node_modules/minimatch": { + "version": "5.1.6", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-5.1.6.tgz", + "integrity": "sha512-lKwV/1brpG6mBUFHtb7NUmtABCb2WZZmm2wNiOA5hAb8VdCS4B3dtMWyvcoViccwAW/COERjXLt0zP1zXUN26g==", + "dev": true, + "dependencies": { + "brace-expansion": "^2.0.1" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/readdirp": { + "version": "3.6.0", + "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-3.6.0.tgz", + "integrity": "sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==", + "dev": true, + "dependencies": { + "picomatch": "^2.2.1" + }, + "engines": { + "node": ">=8.10.0" + } + }, + "node_modules/rechoir": { + "version": "0.7.1", + "resolved": "https://registry.npmjs.org/rechoir/-/rechoir-0.7.1.tgz", + "integrity": "sha512-/njmZ8s1wVeR6pjTZ+0nCnv8SpZNRMT2D1RLOJQESlYFDBvwpTA4KWJpZ+sBJ4+vhjILRcK7JIFdGCdxEAAitg==", + "dev": true, + "dependencies": { + "resolve": "^1.9.0" + }, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/reduce-extract": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/reduce-extract/-/reduce-extract-1.0.0.tgz", + "integrity": "sha512-QF8vjWx3wnRSL5uFMyCjDeDc5EBMiryoT9tz94VvgjKfzecHAVnqmXAwQDcr7X4JmLc2cjkjFGCVzhMqDjgR9g==", + "dev": true, + "dependencies": { + "test-value": "^1.0.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/reduce-extract/node_modules/array-back": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-1.0.4.tgz", + "integrity": "sha512-1WxbZvrmyhkNoeYcizokbmh5oiOCIfyvGtcqbK3Ls1v1fKcquzxnQSceOx6tzq7jmai2kFLWIpGND2cLhH6TPw==", + "dev": true, + "dependencies": { + "typical": "^2.6.0" + }, + "engines": { + "node": ">=0.12.0" + } + }, + "node_modules/reduce-extract/node_modules/test-value": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/test-value/-/test-value-1.1.0.tgz", + "integrity": "sha512-wrsbRo7qP+2Je8x8DsK8ovCGyxe3sYfQwOraIY/09A2gFXU9DYKiTF14W4ki/01AEh56kMzAmlj9CaHGDDUBJA==", + "dev": true, + "dependencies": { + "array-back": "^1.0.2", + "typical": "^2.4.2" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/reduce-flatten": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/reduce-flatten/-/reduce-flatten-3.0.1.tgz", + "integrity": "sha512-bYo+97BmUUOzg09XwfkwALt4PQH1M5L0wzKerBt6WLm3Fhdd43mMS89HiT1B9pJIqko/6lWx3OnV4J9f2Kqp5Q==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/reduce-unique": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/reduce-unique/-/reduce-unique-2.0.1.tgz", + "integrity": "sha512-x4jH/8L1eyZGR785WY+ePtyMNhycl1N2XOLxhCbzZFaqF4AXjLzqSxa2UHgJ2ZVR/HHyPOvl1L7xRnW8ye5MdA==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/reduce-without": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/reduce-without/-/reduce-without-1.0.1.tgz", + "integrity": "sha512-zQv5y/cf85sxvdrKPlfcRzlDn/OqKFThNimYmsS3flmkioKvkUGn2Qg9cJVoQiEvdxFGLE0MQER/9fZ9sUqdxg==", + "dev": true, + "dependencies": { + "test-value": "^2.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/reduce-without/node_modules/array-back": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-1.0.4.tgz", + "integrity": "sha512-1WxbZvrmyhkNoeYcizokbmh5oiOCIfyvGtcqbK3Ls1v1fKcquzxnQSceOx6tzq7jmai2kFLWIpGND2cLhH6TPw==", + "dev": true, + "dependencies": { + "typical": "^2.6.0" + }, + "engines": { + "node": ">=0.12.0" + } + }, + "node_modules/reduce-without/node_modules/test-value": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/test-value/-/test-value-2.1.0.tgz", + "integrity": "sha512-+1epbAxtKeXttkGFMTX9H42oqzOTufR1ceCF+GYA5aOmvaPq9wd4PUS8329fn2RRLGNeUkgRLnVpycjx8DsO2w==", + "dev": true, + "dependencies": { + "array-back": "^1.0.3", + "typical": "^2.6.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/regenerator-runtime": { + "version": "0.13.11", + "resolved": "https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.13.11.tgz", + "integrity": "sha512-kY1AZVr2Ra+t+piVaJ4gxaFaReZVH40AKNo7UCX6W+dEwBo/2oZJzqfuN1qLq1oL45o56cPaTXELwrTh8Fpggg==" + }, + "node_modules/registry-auth-token": { + "version": "4.2.2", + "resolved": "https://registry.npmjs.org/registry-auth-token/-/registry-auth-token-4.2.2.tgz", + "integrity": "sha512-PC5ZysNb42zpFME6D/XlIgtNGdTl8bBOCw90xQLVMpzuuubJKYDWFAEuUNc+Cn8Z8724tg2SDhDRrkVEsqfDMg==", + "dev": true, + "dependencies": { + "rc": "1.2.8" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/registry-url": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/registry-url/-/registry-url-5.1.0.tgz", + "integrity": "sha512-8acYXXTI0AkQv6RAOjE3vOaIXZkT9wo4LOFbBKYQEEnnMNBpKqdUrI6S4NT0KPIo/WVvJ5tE/X5LF/TQUf0ekw==", + "dev": true, + "dependencies": { + "rc": "^1.2.8" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/reinterval": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/reinterval/-/reinterval-1.1.0.tgz", + "integrity": "sha512-QIRet3SYrGp0HUHO88jVskiG6seqUGC5iAG7AwI/BV4ypGcuqk9Du6YQBUOUqm9c8pw1eyLoIaONifRua1lsEQ==" + }, + "node_modules/release-zalgo": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/release-zalgo/-/release-zalgo-1.0.0.tgz", + "integrity": "sha512-gUAyHVHPPC5wdqX/LG4LWtRYtgjxyX78oanFNTMMyFEfOqdC54s3eE82imuWKbOeqYht2CrNf64Qb8vgmmtZGA==", + "dev": true, + "dependencies": { + "es6-error": "^4.0.1" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/require-directory": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz", + "integrity": "sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/require-from-string": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", + "integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/require-main-filename": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/require-main-filename/-/require-main-filename-2.0.0.tgz", + "integrity": "sha512-NKN5kMDylKuldxYLSUfrbo5Tuzh4hd+2E8NPPX02mZtn1VuREQToYe/ZdlJy+J3uCpfaiGF05e7B8W0iXbQHmg==", + "dev": true + }, + "node_modules/requizzle": { + "version": "0.2.4", + "resolved": "https://registry.npmjs.org/requizzle/-/requizzle-0.2.4.tgz", + "integrity": "sha512-JRrFk1D4OQ4SqovXOgdav+K8EAhSB/LJZqCz8tbX0KObcdeM15Ss59ozWMBWmmINMagCwmqn4ZNryUGpBsl6Jw==", + "dev": true, + "dependencies": { + "lodash": "^4.17.21" + } + }, + "node_modules/resolve": { + "version": "1.22.1", + "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.1.tgz", + "integrity": "sha512-nBpuuYuY5jFsli/JIs1oldw6fOQCBioohqWZg/2hiaOybXOft4lonv85uDOKXdf8rhyK159cxU5cDcK/NKk8zw==", + "dev": true, + "dependencies": { + "is-core-module": "^2.9.0", + "path-parse": "^1.0.7", + "supports-preserve-symlinks-flag": "^1.0.0" + }, + "bin": { + "resolve": "bin/resolve" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/resolve-alpn": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/resolve-alpn/-/resolve-alpn-1.2.1.tgz", + "integrity": "sha512-0a1F4l73/ZFZOakJnQ3FvkJ2+gSTQWz/r2KE5OdDY0TxPm5h4GkqkWWfM47T7HsbnOtcJVEF4epCVy6u7Q3K+g==" + }, + "node_modules/resolve-dir": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/resolve-dir/-/resolve-dir-1.0.1.tgz", + "integrity": "sha512-R7uiTjECzvOsWSfdM0QKFNBVFcK27aHOUwdvK53BcW8zqnGdYp0Fbj82cy54+2A4P2tFM22J5kRfe1R+lM/1yg==", + "dev": true, + "dependencies": { + "expand-tilde": "^2.0.0", + "global-modules": "^1.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/resolve-from": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/resolve-from/-/resolve-from-4.0.0.tgz", + "integrity": "sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/responselike": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/responselike/-/responselike-2.0.1.tgz", + "integrity": "sha512-4gl03wn3hj1HP3yzgdI7d3lCkF95F21Pz4BPGvKHinyQzALR5CapwC8yIi0Rh58DEMQ/SguC03wFj2k0M/mHhw==", + "dependencies": { + "lowercase-keys": "^2.0.0" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/retry": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/retry/-/retry-0.6.1.tgz", + "integrity": "sha512-txv1qsctZq8ei9J/uCXgaKKFPjlBB0H2hvtnzw9rjKWFNUFtKh59WprXxpAeAey3/QeWwHdxMFqStPaOAgy+dA==", + "engines": { + "node": "*" + } + }, + "node_modules/rfdc": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/rfdc/-/rfdc-1.3.0.tgz", + "integrity": "sha512-V2hovdzFbOi77/WajaSMXk2OLm+xNIeQdMMuB7icj7bk6zi2F8GGAxigcnDFpJHbNyNcgyJDiP+8nOrY5cZGrA==" + }, + "node_modules/rimraf": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-3.0.2.tgz", + "integrity": "sha512-JZkJMZkAGFFPP2YqXZXPbMlMBgsxzE8ILs4lMIX/2o0L9UBw9O/Y3o6wFw/i9YLapcUJWwqbi3kdxIPdC62TIA==", + "devOptional": true, + "dependencies": { + "glob": "^7.1.3" + }, + "bin": { + "rimraf": "bin.js" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/safe-buffer": { + "version": "5.1.2", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.2.tgz", + "integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==" + }, + "node_modules/safe-json-parse": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/safe-json-parse/-/safe-json-parse-1.0.1.tgz", + "integrity": "sha512-o0JmTu17WGUaUOHa1l0FPGXKBfijbxK6qoHzlkihsDXxzBHvJcA7zgviKR92Xs841rX9pK16unfphLq0/KqX7A==", + "dev": true + }, + "node_modules/safer-buffer": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", + "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==" + }, + "node_modules/sass": { + "version": "1.49.7", + "resolved": "https://registry.npmjs.org/sass/-/sass-1.49.7.tgz", + "integrity": "sha512-13dml55EMIR2rS4d/RDHHP0sXMY3+30e1TKsyXaSz3iLWVoDWEoboY8WzJd5JMnxrRHffKO3wq2mpJ0jxRJiEQ==", + "dev": true, + "dependencies": { + "chokidar": ">=3.0.0 <4.0.0", + "immutable": "^4.0.0", + "source-map-js": ">=0.6.2 <2.0.0" + }, + "bin": { + "sass": "sass.js" + }, + "engines": { + "node": ">=12.0.0" + } + }, + "node_modules/sax": { + "version": "1.2.4", + "resolved": "https://registry.npmjs.org/sax/-/sax-1.2.4.tgz", + "integrity": "sha512-NqVDv9TpANUjFm0N8uM5GxL36UgKi9/atZw+x7YFnQ8ckwFGKrl4xX4yWtrey3UJm5nP1kUbnYgLopqWNSRhWw==" + }, + "node_modules/semver": { + "version": "7.3.5", + "resolved": "https://registry.npmjs.org/semver/-/semver-7.3.5.tgz", + "integrity": "sha512-PoeGJYh8HK4BTO/a9Tf6ZG3veo/A7ZVsYrSA6J8ny9nb3B1VrpkuN+z9OE5wfE5p6H4LchYZsegiQgbJD94ZFQ==", + "dependencies": { + "lru-cache": "^6.0.0" + }, + "bin": { + "semver": "bin/semver.js" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/semver-diff": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/semver-diff/-/semver-diff-3.1.1.tgz", + "integrity": "sha512-GX0Ix/CJcHyB8c4ykpHGIAvLyOwOobtM/8d+TQkAd81/bEjgPHrfba41Vpesr7jX/t8Uh+R3EX9eAS5be+jQYg==", + "dev": true, + "dependencies": { + "semver": "^6.3.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/semver-diff/node_modules/semver": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/semver/-/semver-6.3.0.tgz", + "integrity": "sha512-b39TBaTSfV6yBrapU89p5fKekE2m/NwnDocOVruQFS1/veMgdzuPcnOM34M6CwxW8jH/lxEa5rBoDeUwu5HHTw==", + "dev": true, + "bin": { + "semver": "bin/semver.js" + } + }, + "node_modules/semver/node_modules/lru-cache": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-6.0.0.tgz", + "integrity": "sha512-Jo6dJ04CmSjuznwJSS3pUeWmd/H0ffTlkXXgwZi+eq1UCmqQwCh+eLsYOYCwY991i2Fah4h1BEMCx4qThGbsiA==", + "dependencies": { + "yallist": "^4.0.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/semver/node_modules/yallist": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz", + "integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==" + }, + "node_modules/send": { + "version": "0.17.2", + "resolved": "https://registry.npmjs.org/send/-/send-0.17.2.tgz", + "integrity": "sha512-UJYB6wFSJE3G00nEivR5rgWp8c2xXvJ3OPWPhmuteU0IKj8nKbG3DrjiOmLwpnHGYWAVwA69zmTm++YG0Hmwww==", + "dependencies": { + "debug": "2.6.9", + "depd": "~1.1.2", + "destroy": "~1.0.4", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "fresh": "0.5.2", + "http-errors": "1.8.1", + "mime": "1.6.0", + "ms": "2.1.3", + "on-finished": "~2.3.0", + "range-parser": "~1.2.1", + "statuses": "~1.5.0" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/send/node_modules/mime": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz", + "integrity": "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==", + "bin": { + "mime": "cli.js" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/send/node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==" + }, + "node_modules/serialize-javascript": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-6.0.0.tgz", + "integrity": "sha512-Qr3TosvguFt8ePWqsvRfrKyQXIiW+nGbYpy8XK24NQHE83caxWt+mIymTT19DGFbNWNLfEwsrkSmN64lVWB9ag==", + "dev": true, + "dependencies": { + "randombytes": "^2.1.0" + } + }, + "node_modules/serve-static": { + "version": "1.14.2", + "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.14.2.tgz", + "integrity": "sha512-+TMNA9AFxUEGuC0z2mevogSnn9MXKb4fa7ngeRMJaaGv8vTwnIEkKi+QGvPt33HSnf8pRS+WGM0EbMtCJLKMBQ==", + "dependencies": { + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "parseurl": "~1.3.3", + "send": "0.17.2" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "node_modules/set-blocking": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/set-blocking/-/set-blocking-2.0.0.tgz", + "integrity": "sha512-KiKBS8AnWGEyLzofFfmvKwpdPzqiy16LvQfK3yv/fVH7Bj13/wl3JSR1J+rfgRE9q7xUJK4qvgS8raSOeLUehw==", + "devOptional": true + }, + "node_modules/setprototypeof": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz", + "integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==" + }, + "node_modules/shebang-command": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz", + "integrity": "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==", + "dev": true, + "dependencies": { + "shebang-regex": "^3.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/shebang-regex": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz", + "integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/shelljs": { + "version": "0.5.3", + "resolved": "https://registry.npmjs.org/shelljs/-/shelljs-0.5.3.tgz", + "integrity": "sha512-C2FisSSW8S6TIYHHiMHN0NqzdjWfTekdMpA2FJTbRWnQMLO1RRIXEB9eVZYOlofYmjZA7fY3ChoFu09MeI3wlQ==", + "dev": true, + "bin": { + "shjs": "bin/shjs" + }, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/should": { + "version": "13.2.3", + "resolved": "https://registry.npmjs.org/should/-/should-13.2.3.tgz", + "integrity": "sha512-ggLesLtu2xp+ZxI+ysJTmNjh2U0TsC+rQ/pfED9bUZZ4DKefP27D+7YJVVTvKsmjLpIi9jAa7itwDGkDDmt1GQ==", + "dev": true, + "dependencies": { + "should-equal": "^2.0.0", + "should-format": "^3.0.3", + "should-type": "^1.4.0", + "should-type-adaptors": "^1.0.1", + "should-util": "^1.0.0" + } + }, + "node_modules/should-equal": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/should-equal/-/should-equal-2.0.0.tgz", + "integrity": "sha512-ZP36TMrK9euEuWQYBig9W55WPC7uo37qzAEmbjHz4gfyuXrEUgF8cUvQVO+w+d3OMfPvSRQJ22lSm8MQJ43LTA==", + "dev": true, + "dependencies": { + "should-type": "^1.4.0" + } + }, + "node_modules/should-format": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/should-format/-/should-format-3.0.3.tgz", + "integrity": "sha512-hZ58adtulAk0gKtua7QxevgUaXTTXxIi8t41L3zo9AHvjXO1/7sdLECuHeIN2SRtYXpNkmhoUP2pdeWgricQ+Q==", + "dev": true, + "dependencies": { + "should-type": "^1.3.0", + "should-type-adaptors": "^1.0.1" + } + }, + "node_modules/should-sinon": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/should-sinon/-/should-sinon-0.0.6.tgz", + "integrity": "sha512-ScBOH5uW5QVFaONmUnIXANSR6z5B8IKzEmBP3HE5sPOCDuZ88oTMdUdnKoCVQdLcCIrRrhRLPS5YT+7H40a04g==", + "dev": true, + "peerDependencies": { + "should": ">= 8.x" + } + }, + "node_modules/should-type": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/should-type/-/should-type-1.4.0.tgz", + "integrity": "sha512-MdAsTu3n25yDbIe1NeN69G4n6mUnJGtSJHygX3+oN0ZbO3DTiATnf7XnYJdGT42JCXurTb1JI0qOBR65shvhPQ==", + "dev": true + }, + "node_modules/should-type-adaptors": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/should-type-adaptors/-/should-type-adaptors-1.1.0.tgz", + "integrity": "sha512-JA4hdoLnN+kebEp2Vs8eBe9g7uy0zbRo+RMcU0EsNy+R+k049Ki+N5tT5Jagst2g7EAja+euFuoXFCa8vIklfA==", + "dev": true, + "dependencies": { + "should-type": "^1.3.0", + "should-util": "^1.0.0" + } + }, + "node_modules/should-util": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/should-util/-/should-util-1.0.1.tgz", + "integrity": "sha512-oXF8tfxx5cDk8r2kYqlkUJzZpDBqVY/II2WhvU0n9Y3XYvAYRmeaf1PvvIvTgPnv4KJ+ES5M0PyDq5Jp+Ygy2g==", + "dev": true + }, + "node_modules/side-channel": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.0.4.tgz", + "integrity": "sha512-q5XPytqFEIKHkGdiMIrY10mvLRvnQh42/+GoBlFW3b2LXLE2xxJpZFdm94we0BaoV3RwJyGqg5wS7epxTv0Zvw==", + "dev": true, + "dependencies": { + "call-bind": "^1.0.0", + "get-intrinsic": "^1.0.2", + "object-inspect": "^1.9.0" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/signal-exit": { + "version": "3.0.7", + "resolved": "https://registry.npmjs.org/signal-exit/-/signal-exit-3.0.7.tgz", + "integrity": "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ==", + "devOptional": true + }, + "node_modules/simple-cli": { + "version": "5.0.5", + "resolved": "https://registry.npmjs.org/simple-cli/-/simple-cli-5.0.5.tgz", + "integrity": "sha512-Er2FhsIayL/sktxg6fOCdNQJBTXhlf/fswNFsdmks88xsHzQ/IXGwxYgSSKeXBq4yqn83/iD4Sg8yjagwysUgw==", + "dev": true, + "dependencies": { + "async": "^3.1.0", + "chalk": "^2.4.2", + "cross-spawn": "^7.0.0", + "key-list": "^0.1.4", + "lodash": "^4.17.15", + "opted": "^1.0.0" + } + }, + "node_modules/simple-cli/node_modules/ansi-styles": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz", + "integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==", + "dev": true, + "dependencies": { + "color-convert": "^1.9.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/simple-cli/node_modules/async": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/async/-/async-3.2.4.tgz", + "integrity": "sha512-iAB+JbDEGXhyIUavoDl9WP/Jj106Kz9DEn1DPgYw5ruDn0e3Wgi3sKFm55sASdGBNOQB8F59d9qQ7deqrHA8wQ==", + "dev": true + }, + "node_modules/simple-cli/node_modules/chalk": { + "version": "2.4.2", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz", + "integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==", + "dev": true, + "dependencies": { + "ansi-styles": "^3.2.1", + "escape-string-regexp": "^1.0.5", + "supports-color": "^5.3.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/simple-cli/node_modules/color-convert": { + "version": "1.9.3", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.3.tgz", + "integrity": "sha512-QfAUtd+vFdAtFQcC8CCyYt1fYWxSqAiK2cSD6zDB8N3cpsEBAvRxp9zOGg6G/SHHJYAT88/az/IuDGALsNVbGg==", + "dev": true, + "dependencies": { + "color-name": "1.1.3" + } + }, + "node_modules/simple-cli/node_modules/color-name": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz", + "integrity": "sha512-72fSenhMw2HZMTVHeCA9KCmpEIbzWiQsjN+BHcBbS9vr1mtt+vJjPdksIBNUmKAW8TFUDPJK5SUU3QhE9NEXDw==", + "dev": true + }, + "node_modules/simple-cli/node_modules/escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/simple-cli/node_modules/has-flag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", + "integrity": "sha512-sKJf1+ceQBr4SMkvQnBDNDtf4TXpVhVGateu0t918bl30FnbE2m4vNLX+VWe/dpjlb+HugGYzW7uQXH98HPEYw==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/simple-cli/node_modules/supports-color": { + "version": "5.5.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz", + "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==", + "dev": true, + "dependencies": { + "has-flag": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/sinon": { + "version": "11.1.2", + "resolved": "https://registry.npmjs.org/sinon/-/sinon-11.1.2.tgz", + "integrity": "sha512-59237HChms4kg7/sXhiRcUzdSkKuydDeTiamT/jesUVHshBgL8XAmhgFo0GfK6RruMDM/iRSij1EybmMog9cJw==", + "dev": true, + "dependencies": { + "@sinonjs/commons": "^1.8.3", + "@sinonjs/fake-timers": "^7.1.2", + "@sinonjs/samsam": "^6.0.2", + "diff": "^5.0.0", + "nise": "^5.1.0", + "supports-color": "^7.2.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/sinon" + } + }, + "node_modules/sort-array": { + "version": "4.1.5", + "resolved": "https://registry.npmjs.org/sort-array/-/sort-array-4.1.5.tgz", + "integrity": "sha512-Ya4peoS1fgFN42RN1REk2FgdNOeLIEMKFGJvs7VTP3OklF8+kl2SkpVliZ4tk/PurWsrWRsdNdU+tgyOBkB9sA==", + "dev": true, + "dependencies": { + "array-back": "^5.0.0", + "typical": "^6.0.1" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/sort-array/node_modules/array-back": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-5.0.0.tgz", + "integrity": "sha512-kgVWwJReZWmVuWOQKEOohXKJX+nD02JAZ54D1RRWlv8L0NebauKAaFxACKzB74RTclt1+WNz5KHaLRDAPZbDEw==", + "dev": true, + "engines": { + "node": ">=10" + } + }, + "node_modules/sort-array/node_modules/typical": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/typical/-/typical-6.0.1.tgz", + "integrity": "sha512-+g3NEp7fJLe9DPa1TArHm9QAA7YciZmWnfAqEaFrBihQ7epOv9i99rjtgb6Iz0wh3WuQDjsCTDfgRoGnmHN81A==", + "dev": true, + "engines": { + "node": ">=10" + } + }, + "node_modules/source-map": { + "version": "0.5.7", + "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.5.7.tgz", + "integrity": "sha512-LbrmJOMUSdEVxIKvdcJzQC+nQhe8FUZQTXQy6+I75skNgn3OoQ0DZA8YnFa7gp8tqtL3KPf1kmo0R5DoApeSGQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/source-map-js": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.0.2.tgz", + "integrity": "sha512-R0XvVJ9WusLiqTCEiGCmICCMplcCkIwwR11mOSD9CR5u+IXYdiseeEuXCVAjS54zqwkLcPNnmU4OeJ6tUrWhDw==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/spawn-wrap": { + "version": "1.4.3", + "resolved": "https://registry.npmjs.org/spawn-wrap/-/spawn-wrap-1.4.3.tgz", + "integrity": "sha512-IgB8md0QW/+tWqcavuFgKYR/qIRvJkRLPJDFaoXtLLUaVcCDK0+HeFTkmQHj3eprcYhc+gOl0aEA1w7qZlYezw==", + "dev": true, + "dependencies": { + "foreground-child": "^1.5.6", + "mkdirp": "^0.5.0", + "os-homedir": "^1.0.1", + "rimraf": "^2.6.2", + "signal-exit": "^3.0.2", + "which": "^1.3.0" + } + }, + "node_modules/spawn-wrap/node_modules/mkdirp": { + "version": "0.5.6", + "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.6.tgz", + "integrity": "sha512-FP+p8RB8OWpF3YZBCrP5gtADmtXApB5AMLn+vdyA+PyxCjrCs00mjyUozssO33cwDeT3wNGdLxJ5M//YqtHAJw==", + "dev": true, + "dependencies": { + "minimist": "^1.2.6" + }, + "bin": { + "mkdirp": "bin/cmd.js" + } + }, + "node_modules/spawn-wrap/node_modules/rimraf": { + "version": "2.7.1", + "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-2.7.1.tgz", + "integrity": "sha512-uWjbaKIK3T1OSVptzX7Nl6PvQ3qAGtKEtVRjRuazjfL3Bx5eI409VZSqgND+4UNnmzLVdPj9FqFJNPqBZFve4w==", + "dev": true, + "dependencies": { + "glob": "^7.1.3" + }, + "bin": { + "rimraf": "bin.js" + } + }, + "node_modules/spawn-wrap/node_modules/which": { + "version": "1.3.1", + "resolved": "https://registry.npmjs.org/which/-/which-1.3.1.tgz", + "integrity": "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==", + "dev": true, + "dependencies": { + "isexe": "^2.0.0" + }, + "bin": { + "which": "bin/which" + } + }, + "node_modules/spdx-correct": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/spdx-correct/-/spdx-correct-3.2.0.tgz", + "integrity": "sha512-kN9dJbvnySHULIluDHy32WHRUu3Og7B9sbY7tsFLctQkIqnMh3hErYgdMjTYuqmcXX+lK5T1lnUt3G7zNswmZA==", + "dev": true, + "dependencies": { + "spdx-expression-parse": "^3.0.0", + "spdx-license-ids": "^3.0.0" + } + }, + "node_modules/spdx-exceptions": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/spdx-exceptions/-/spdx-exceptions-2.3.0.tgz", + "integrity": "sha512-/tTrYOC7PPI1nUAgx34hUpqXuyJG+DTHJTnIULG4rDygi4xu/tfgmq1e1cIRwRzwZgo4NLySi+ricLkZkw4i5A==", + "dev": true + }, + "node_modules/spdx-expression-parse": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/spdx-expression-parse/-/spdx-expression-parse-3.0.1.tgz", + "integrity": "sha512-cbqHunsQWnJNE6KhVSMsMeH5H/L9EpymbzqTQ3uLwNCLZ1Q481oWaofqH7nO6V07xlXwY6PhQdQ2IedWx/ZK4Q==", + "dev": true, + "dependencies": { + "spdx-exceptions": "^2.1.0", + "spdx-license-ids": "^3.0.0" + } + }, + "node_modules/spdx-license-ids": { + "version": "3.0.13", + "resolved": "https://registry.npmjs.org/spdx-license-ids/-/spdx-license-ids-3.0.13.tgz", + "integrity": "sha512-XkD+zwiqXHikFZm4AX/7JSCXA98U5Db4AFd5XUg/+9UNtnH75+Z9KxtpYiJZx36mUDVOwH83pl7yvCer6ewM3w==", + "dev": true + }, + "node_modules/split2": { + "version": "3.2.2", + "resolved": "https://registry.npmjs.org/split2/-/split2-3.2.2.tgz", + "integrity": "sha512-9NThjpgZnifTkJpzTZ7Eue85S49QwpNhZTq6GRJwObb6jnLFNGB7Qm73V5HewTROPyxD0C29xqmaI68bQtV+hg==", + "dependencies": { + "readable-stream": "^3.0.0" + } + }, + "node_modules/sprintf-js": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz", + "integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g==" + }, + "node_modules/statuses": { + "version": "1.5.0", + "resolved": "https://registry.npmjs.org/statuses/-/statuses-1.5.0.tgz", + "integrity": "sha512-OpZ3zP+jT1PI7I8nemJX4AKmAX070ZkYPVWV/AaKTJl+tXCTGyVdC1a4SL8RUQYEwk/f34ZX8UTykN68FwrqAA==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/stoppable": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/stoppable/-/stoppable-1.1.0.tgz", + "integrity": "sha512-KXDYZ9dszj6bzvnEMRYvxgeTHU74QBFL54XKtP3nyMuJ81CFYtABZ3bAzL2EdFUaEwJOBOgENyFj3R7oTzDyyw==", + "dev": true, + "engines": { + "node": ">=4", + "npm": ">=6" + } + }, + "node_modules/stream-buffers": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/stream-buffers/-/stream-buffers-3.0.2.tgz", + "integrity": "sha512-DQi1h8VEBA/lURbSwFtEHnSTb9s2/pwLEaFuNhXwy1Dx3Sa0lOuYT2yNUr4/j2fs8oCAMANtrZ5OrPZtyVs3MQ==", + "dev": true, + "engines": { + "node": ">= 0.10.0" + } + }, + "node_modules/stream-connect": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/stream-connect/-/stream-connect-1.0.2.tgz", + "integrity": "sha512-68Kl+79cE0RGKemKkhxTSg8+6AGrqBt+cbZAXevg2iJ6Y3zX4JhA/sZeGzLpxW9cXhmqAcE7KnJCisUmIUfnFQ==", + "dev": true, + "dependencies": { + "array-back": "^1.0.2" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/stream-connect/node_modules/array-back": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-1.0.4.tgz", + "integrity": "sha512-1WxbZvrmyhkNoeYcizokbmh5oiOCIfyvGtcqbK3Ls1v1fKcquzxnQSceOx6tzq7jmai2kFLWIpGND2cLhH6TPw==", + "dev": true, + "dependencies": { + "typical": "^2.6.0" + }, + "engines": { + "node": ">=0.12.0" + } + }, + "node_modules/stream-shift": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/stream-shift/-/stream-shift-1.0.1.tgz", + "integrity": "sha512-AiisoFqQ0vbGcZgQPY1cdP2I76glaVA/RauYR4G4thNFgkTqr90yXTo4LYX60Jl+sIlPNHHdGSwo01AvbKUSVQ==" + }, + "node_modules/stream-via": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/stream-via/-/stream-via-1.0.4.tgz", + "integrity": "sha512-DBp0lSvX5G9KGRDTkR/R+a29H+Wk2xItOF+MpZLLNDWbEV9tGPnqLPxHEYjmiz8xGtJHRIqmI+hCjmNzqoA4nQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/streamsearch": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/streamsearch/-/streamsearch-0.1.2.tgz", + "integrity": "sha512-jos8u++JKm0ARcSUTAZXOVC0mSox7Bhn6sBgty73P1f3JGf7yG2clTbBNHUdde/kdvP2FESam+vM6l8jBrNxHA==", + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/string_decoder": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz", + "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==", + "dependencies": { + "safe-buffer": "~5.2.0" + } + }, + "node_modules/string_decoder/node_modules/safe-buffer": { + "version": "5.2.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", + "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ] + }, + "node_modules/string-template": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/string-template/-/string-template-0.2.1.tgz", + "integrity": "sha512-Yptehjogou2xm4UJbxJ4CxgZx12HBfeystp0y3x7s4Dj32ltVVG1Gg8YhKjHZkHicuKpZX/ffilA8505VbUbpw==", + "dev": true + }, + "node_modules/string-width": { + "version": "4.2.3", + "resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz", + "integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==", + "devOptional": true, + "dependencies": { + "emoji-regex": "^8.0.0", + "is-fullwidth-code-point": "^3.0.0", + "strip-ansi": "^6.0.1" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/strip-ansi": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz", + "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==", + "devOptional": true, + "dependencies": { + "ansi-regex": "^5.0.1" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/strip-bom": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/strip-bom/-/strip-bom-3.0.0.tgz", + "integrity": "sha512-vavAMRXOgBVNF6nyEEmL3DBK19iRpDcoIwW+swQ+CbGiu7lju6t+JklA1MHweoWtadgt4ISVUsXLyDq34ddcwA==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/strip-json-comments": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-3.1.1.tgz", + "integrity": "sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig==", + "dev": true, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/superagent": { + "version": "7.1.5", + "resolved": "https://registry.npmjs.org/superagent/-/superagent-7.1.5.tgz", + "integrity": "sha512-HQYyGuDRFGmZ6GNC4hq2f37KnsY9Lr0/R1marNZTgMweVDQLTLJJ6DGQ9Tj/xVVs5HEnop9EMmTbywb5P30aqw==", + "dev": true, + "dependencies": { + "component-emitter": "^1.3.0", + "cookiejar": "^2.1.3", + "debug": "^4.3.4", + "fast-safe-stringify": "^2.1.1", + "form-data": "^4.0.0", + "formidable": "^2.0.1", + "methods": "^1.1.2", + "mime": "^2.5.0", + "qs": "^6.10.3", + "readable-stream": "^3.6.0", + "semver": "^7.3.7" + }, + "engines": { + "node": ">=6.4.0 <13 || >=14" + } + }, + "node_modules/superagent/node_modules/debug": { + "version": "4.3.4", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz", + "integrity": "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ==", + "dev": true, + "dependencies": { + "ms": "2.1.2" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/superagent/node_modules/lru-cache": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-6.0.0.tgz", + "integrity": "sha512-Jo6dJ04CmSjuznwJSS3pUeWmd/H0ffTlkXXgwZi+eq1UCmqQwCh+eLsYOYCwY991i2Fah4h1BEMCx4qThGbsiA==", + "dev": true, + "dependencies": { + "yallist": "^4.0.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/superagent/node_modules/mime": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/mime/-/mime-2.6.0.tgz", + "integrity": "sha512-USPkMeET31rOMiarsBNIHZKLGgvKc/LrjofAnBlOttf5ajRvqiRA8QsenbcooctK6d6Ts6aqZXBA+XbkKthiQg==", + "dev": true, + "bin": { + "mime": "cli.js" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/superagent/node_modules/ms": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz", + "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==", + "dev": true + }, + "node_modules/superagent/node_modules/qs": { + "version": "6.11.1", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.11.1.tgz", + "integrity": "sha512-0wsrzgTz/kAVIeuxSjnpGC56rzYtr6JT/2BwEvMaPhFIoYa1aGO8LbzuU1R0uUYQkLpWBTOj0l/CLAJB64J6nQ==", + "dev": true, + "dependencies": { + "side-channel": "^1.0.4" + }, + "engines": { + "node": ">=0.6" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/superagent/node_modules/semver": { + "version": "7.3.8", + "resolved": "https://registry.npmjs.org/semver/-/semver-7.3.8.tgz", + "integrity": "sha512-NB1ctGL5rlHrPJtFDVIVzTyQylMLu9N9VICA6HSFJo8MCGVTMW6gfpicwKmmK/dAjTOrqu5l63JJOpDSrAis3A==", + "dev": true, + "dependencies": { + "lru-cache": "^6.0.0" + }, + "bin": { + "semver": "bin/semver.js" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/superagent/node_modules/yallist": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz", + "integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==", + "dev": true + }, + "node_modules/supertest": { + "version": "6.2.2", + "resolved": "https://registry.npmjs.org/supertest/-/supertest-6.2.2.tgz", + "integrity": "sha512-wCw9WhAtKJsBvh07RaS+/By91NNE0Wh0DN19/hWPlBOU8tAfOtbZoVSV4xXeoKoxgPx0rx2y+y+8660XtE7jzg==", + "dev": true, + "dependencies": { + "methods": "^1.1.2", + "superagent": "^7.1.0" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/supports-color": { + "version": "7.2.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz", + "integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==", + "dev": true, + "dependencies": { + "has-flag": "^4.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/supports-preserve-symlinks-flag": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/supports-preserve-symlinks-flag/-/supports-preserve-symlinks-flag-1.0.0.tgz", + "integrity": "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w==", + "dev": true, + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/table-layout": { + "version": "0.4.5", + "resolved": "https://registry.npmjs.org/table-layout/-/table-layout-0.4.5.tgz", + "integrity": "sha512-zTvf0mcggrGeTe/2jJ6ECkJHAQPIYEwDoqsiqBjI24mvRmQbInK5jq33fyypaCBxX08hMkfmdOqj6haT33EqWw==", + "dev": true, + "dependencies": { + "array-back": "^2.0.0", + "deep-extend": "~0.6.0", + "lodash.padend": "^4.6.1", + "typical": "^2.6.1", + "wordwrapjs": "^3.0.0" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/table-layout/node_modules/array-back": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-2.0.0.tgz", + "integrity": "sha512-eJv4pLLufP3g5kcZry0j6WXpIbzYw9GUB4mVJZno9wfwiBxbizTnHCw3VJb07cBihbFX48Y7oSrW9y+gt4glyw==", + "dev": true, + "dependencies": { + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/taffydb": { + "version": "2.6.2", + "resolved": "https://registry.npmjs.org/taffydb/-/taffydb-2.6.2.tgz", + "integrity": "sha512-y3JaeRSplks6NYQuCOj3ZFMO3j60rTwbuKCvZxsAraGYH2epusatvZ0baZYA01WsGqJBq/Dl6vOrMUJqyMj8kA==", + "dev": true + }, + "node_modules/tar": { + "version": "6.1.11", + "resolved": "https://registry.npmjs.org/tar/-/tar-6.1.11.tgz", + "integrity": "sha512-an/KZQzQUkZCkuoAA64hM92X0Urb6VpRhAFllDzz44U2mcD5scmT3zBc4VgVpkugF580+DQn8eAFSyoQt0tznA==", + "dependencies": { + "chownr": "^2.0.0", + "fs-minipass": "^2.0.0", + "minipass": "^3.0.0", + "minizlib": "^2.1.1", + "mkdirp": "^1.0.3", + "yallist": "^4.0.0" + }, + "engines": { + "node": ">= 10" + } + }, + "node_modules/tar-stream": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz", + "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==", + "dev": true, + "dependencies": { + "bl": "^4.0.3", + "end-of-stream": "^1.4.1", + "fs-constants": "^1.0.0", + "inherits": "^2.0.3", + "readable-stream": "^3.1.1" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/tar/node_modules/yallist": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz", + "integrity": "sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==" + }, + "node_modules/temp-path": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/temp-path/-/temp-path-1.0.0.tgz", + "integrity": "sha512-TvmyH7kC6ZVTYkqCODjJIbgvu0FKiwQpZ4D1aknE7xpcDf/qEOB8KZEK5ef2pfbVoiBhNWs3yx4y+ESMtNYmlg==", + "dev": true + }, + "node_modules/test-exclude": { + "version": "5.2.3", + "resolved": "https://registry.npmjs.org/test-exclude/-/test-exclude-5.2.3.tgz", + "integrity": "sha512-M+oxtseCFO3EDtAaGH7iiej3CBkzXqFMbzqYAACdzKui4eZA+pq3tZEwChvOdNfa7xxy8BfbmgJSIr43cC/+2g==", + "dev": true, + "dependencies": { + "glob": "^7.1.3", + "minimatch": "^3.0.4", + "read-pkg-up": "^4.0.0", + "require-main-filename": "^2.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/test-exclude/node_modules/find-up": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/find-up/-/find-up-3.0.0.tgz", + "integrity": "sha512-1yD6RmLI1XBfxugvORwlck6f75tYL+iR0jqwsOrOxMZyGYqUuDhJ0l4AXdO1iX/FTs9cBAMEk1gWSEx1kSbylg==", + "dev": true, + "dependencies": { + "locate-path": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/test-exclude/node_modules/locate-path": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/locate-path/-/locate-path-3.0.0.tgz", + "integrity": "sha512-7AO748wWnIhNqAuaty2ZWHkQHRSNfPVIsPIfwEOWO22AmaoVrWavlOcMR5nzTLNYvp36X220/maaRsrec1G65A==", + "dev": true, + "dependencies": { + "p-locate": "^3.0.0", + "path-exists": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/test-exclude/node_modules/p-limit": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/p-limit/-/p-limit-2.3.0.tgz", + "integrity": "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w==", + "dev": true, + "dependencies": { + "p-try": "^2.0.0" + }, + "engines": { + "node": ">=6" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/test-exclude/node_modules/p-locate": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/p-locate/-/p-locate-3.0.0.tgz", + "integrity": "sha512-x+12w/To+4GFfgJhBEpiDcLozRJGegY+Ei7/z0tSLkMmxGZNybVMSfWj9aJn8Z5Fc7dBUNJOOVgPv2H7IwulSQ==", + "dev": true, + "dependencies": { + "p-limit": "^2.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/test-exclude/node_modules/path-exists": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/path-exists/-/path-exists-3.0.0.tgz", + "integrity": "sha512-bpC7GYwiDYQ4wYLe+FA8lhRjhQCMcQGuSgGGqDkg/QerRWw9CmGRT0iSOVRSZJ29NMLZgIzqaljJ63oaL4NIJQ==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/test-exclude/node_modules/read-pkg": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/read-pkg/-/read-pkg-3.0.0.tgz", + "integrity": "sha512-BLq/cCO9two+lBgiTYNqD6GdtK8s4NpaWrl6/rCO9w0TUS8oJl7cmToOZfRYllKTISY6nt1U7jQ53brmKqY6BA==", + "dev": true, + "dependencies": { + "load-json-file": "^4.0.0", + "normalize-package-data": "^2.3.2", + "path-type": "^3.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/test-exclude/node_modules/read-pkg-up": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/read-pkg-up/-/read-pkg-up-4.0.0.tgz", + "integrity": "sha512-6etQSH7nJGsK0RbG/2TeDzZFa8shjQ1um+SwQQ5cwKy0dhSXdOncEhb1CPpvQG4h7FyOV6EB6YlV0yJvZQNAkA==", + "dev": true, + "dependencies": { + "find-up": "^3.0.0", + "read-pkg": "^3.0.0" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/test-value": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/test-value/-/test-value-3.0.0.tgz", + "integrity": "sha512-sVACdAWcZkSU9x7AOmJo5TqE+GyNJknHaHsMrR6ZnhjVlVN9Yx6FjHrsKZ3BjIpPCT68zYesPWkakrNupwfOTQ==", + "dev": true, + "dependencies": { + "array-back": "^2.0.0", + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/test-value/node_modules/array-back": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/array-back/-/array-back-2.0.0.tgz", + "integrity": "sha512-eJv4pLLufP3g5kcZry0j6WXpIbzYw9GUB4mVJZno9wfwiBxbizTnHCw3VJb07cBihbFX48Y7oSrW9y+gt4glyw==", + "dev": true, + "dependencies": { + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/through2": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/through2/-/through2-2.0.5.tgz", + "integrity": "sha512-/mrRod8xqpA+IHSLyGCQ2s8SPHiCDEeQJSep1jqLYeEUClOFG2Qsh+4FU6G9VeqpZnGW/Su8LQGc4YKni5rYSQ==", + "dev": true, + "dependencies": { + "readable-stream": "~2.3.6", + "xtend": "~4.0.1" + } + }, + "node_modules/through2/node_modules/readable-stream": { + "version": "2.3.8", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.8.tgz", + "integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==", + "dev": true, + "dependencies": { + "core-util-is": "~1.0.0", + "inherits": "~2.0.3", + "isarray": "~1.0.0", + "process-nextick-args": "~2.0.0", + "safe-buffer": "~5.1.1", + "string_decoder": "~1.1.1", + "util-deprecate": "~1.0.1" + } + }, + "node_modules/through2/node_modules/string_decoder": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz", + "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==", + "dev": true, + "dependencies": { + "safe-buffer": "~5.1.0" + } + }, + "node_modules/tiny-lr": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/tiny-lr/-/tiny-lr-1.1.1.tgz", + "integrity": "sha512-44yhA3tsaRoMOjQQ+5v5mVdqef+kH6Qze9jTpqtVufgYjYt08zyZAwNwwVBj3i1rJMnR52IxOW0LK0vBzgAkuA==", + "dev": true, + "dependencies": { + "body": "^5.1.0", + "debug": "^3.1.0", + "faye-websocket": "~0.10.0", + "livereload-js": "^2.3.0", + "object-assign": "^4.1.0", + "qs": "^6.4.0" + } + }, + "node_modules/tiny-lr/node_modules/debug": { + "version": "3.2.7", + "resolved": "https://registry.npmjs.org/debug/-/debug-3.2.7.tgz", + "integrity": "sha512-CFjzYYAi4ThfiQvizrFQevTTXHtnCqWfe7x1AhgEscTz6ZbLbfoLRLPugTQyBth6f8ZERVUSyWHFD/7Wu4t1XQ==", + "dev": true, + "dependencies": { + "ms": "^2.1.1" + } + }, + "node_modules/tiny-lr/node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true + }, + "node_modules/to-fast-properties": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/to-fast-properties/-/to-fast-properties-2.0.0.tgz", + "integrity": "sha512-/OaKK0xYrs3DmxRYqL/yDc+FxFUVYhDlXMhRmv3z915w2HF1tnN1omB354j8VUGO/hbRzyD6Y3sA7v7GS/ceog==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/to-readable-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/to-readable-stream/-/to-readable-stream-1.0.0.tgz", + "integrity": "sha512-Iq25XBt6zD5npPhlLVXGFN3/gyR2/qODcKNNyTMd4vbm39HUaOiAM4PMq0eMVC/Tkxz+Zjdsc55g9yyz+Yq00Q==", + "dev": true, + "engines": { + "node": ">=6" + } + }, + "node_modules/to-regex-range": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz", + "integrity": "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==", + "dev": true, + "dependencies": { + "is-number": "^7.0.0" + }, + "engines": { + "node": ">=8.0" + } + }, + "node_modules/toidentifier": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz", + "integrity": "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==", + "engines": { + "node": ">=0.6" + } + }, + "node_modules/touch": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/touch/-/touch-3.1.0.tgz", + "integrity": "sha512-WBx8Uy5TLtOSRtIq+M03/sKDrXCLHxwDcquSP2c43Le03/9serjQBIztjRz6FkJez9D/hleyAXTBGLwwZUw9lA==", + "dev": true, + "dependencies": { + "nopt": "~1.0.10" + }, + "bin": { + "nodetouch": "bin/nodetouch.js" + } + }, + "node_modules/touch/node_modules/nopt": { + "version": "1.0.10", + "resolved": "https://registry.npmjs.org/nopt/-/nopt-1.0.10.tgz", + "integrity": "sha512-NWmpvLSqUrgrAC9HCuxEvb+PSloHpqVu+FqcO4eeF2h5qYRhA7ev6KvelyQAKtegUbC6RypJnlEOhd8vloNKYg==", + "dev": true, + "dependencies": { + "abbrev": "1" + }, + "bin": { + "nopt": "bin/nopt.js" + }, + "engines": { + "node": "*" + } + }, + "node_modules/tough-cookie": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-4.0.0.tgz", + "integrity": "sha512-tHdtEpQCMrc1YLrMaqXXcj6AxhYi/xgit6mZu1+EDWUn+qhUf8wMQoFIy9NXuq23zAwtcB0t/MjACGR18pcRbg==", + "dependencies": { + "psl": "^1.1.33", + "punycode": "^2.1.1", + "universalify": "^0.1.2" + }, + "engines": { + "node": ">=6" + } + }, + "node_modules/tough-cookie/node_modules/universalify": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/universalify/-/universalify-0.1.2.tgz", + "integrity": "sha512-rBJeI5CXAlmy1pV+617WB9J63U6XcazHHF2f2dbJix4XzpUF0RS3Zbj0FGIOCAva5P/d/GBOYaACQ1w+0azUkg==", + "engines": { + "node": ">= 4.0.0" + } + }, + "node_modules/tr46": { + "version": "0.0.3", + "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz", + "integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==", + "devOptional": true + }, + "node_modules/tslib": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.5.0.tgz", + "integrity": "sha512-336iVw3rtn2BUK7ORdIAHTyxHGRIHVReokCR3XjbckJMK7ms8FysBfhLR8IXnAgy7T0PTPNBWKiH514FOW/WSg==" + }, + "node_modules/type-detect": { + "version": "4.0.8", + "resolved": "https://registry.npmjs.org/type-detect/-/type-detect-4.0.8.tgz", + "integrity": "sha512-0fr/mIH1dlO+x7TlcMy+bIDqKPsw/70tVyeHW787goQjhmqaZe10uwLujubK9q9Lg6Fiho1KUKDYz0Z7k7g5/g==", + "dev": true, + "engines": { + "node": ">=4" + } + }, + "node_modules/type-fest": { + "version": "0.8.1", + "resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.8.1.tgz", + "integrity": "sha512-4dbzIzqvjtgiM5rw1k5rEHtBANKmdudhGyBEajN01fEyhaAIhsoKNy6y7+IN93IfpFtwY9iqi7kD+xwKhQsNJA==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/type-is": { + "version": "1.6.18", + "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz", + "integrity": "sha512-TkRKr9sUTxEH8MdfuCSP7VizJyzRNMjj2J2do2Jr3Kym598JVdEksuzPQCnlFPW4ky9Q+iA+ma9BGm06XQBy8g==", + "dependencies": { + "media-typer": "0.3.0", + "mime-types": "~2.1.24" + }, + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/type-is/node_modules/media-typer": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz", + "integrity": "sha512-dq+qelQ9akHpcOl/gUVRTxVIOkAJ1wR3QAvb4RsVjS8oVoFjDGTc679wJYmUmknUF5HwMLOgb5O+a3KxfWapPQ==", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/typedarray": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/typedarray/-/typedarray-0.0.6.tgz", + "integrity": "sha512-/aCDEGatGvZ2BIk+HmLf4ifCJFwvKFNb9/JeZPMulfgFracn9QFcAf5GO8B/mweUjSoblS5In0cWhqpfs/5PQA==" + }, + "node_modules/typedarray-to-buffer": { + "version": "3.1.5", + "resolved": "https://registry.npmjs.org/typedarray-to-buffer/-/typedarray-to-buffer-3.1.5.tgz", + "integrity": "sha512-zdu8XMNEDepKKR+XYOXAVPtWui0ly0NtohUscw+UmaHiAWT8hrV1rr//H6V+0DvJ3OQ19S979M0laLfX8rm82Q==", + "dev": true, + "dependencies": { + "is-typedarray": "^1.0.0" + } + }, + "node_modules/typical": { + "version": "2.6.1", + "resolved": "https://registry.npmjs.org/typical/-/typical-2.6.1.tgz", + "integrity": "sha512-ofhi8kjIje6npGozTip9Fr8iecmYfEbS06i0JnIg+rh51KakryWF4+jX8lLKZVhy6N+ID45WYSFCxPOdTWCzNg==", + "dev": true + }, + "node_modules/uc.micro": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/uc.micro/-/uc.micro-1.0.6.tgz", + "integrity": "sha512-8Y75pvTYkLJW2hWQHXxoqRgV7qb9B+9vFEtidML+7koHUFapnVJAZ6cKs+Qjz5Aw3aZWHMC6u0wJE3At+nSGwA==", + "dev": true + }, + "node_modules/uglify-js": { + "version": "3.15.1", + "resolved": "https://registry.npmjs.org/uglify-js/-/uglify-js-3.15.1.tgz", + "integrity": "sha512-FAGKF12fWdkpvNJZENacOH0e/83eG6JyVQyanIJaBXCN1J11TUQv1T1/z8S+Z0CG0ZPk1nPcreF/c7lrTd0TEQ==", + "bin": { + "uglifyjs": "bin/uglifyjs" + }, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/uid-safe": { + "version": "2.1.5", + "resolved": "https://registry.npmjs.org/uid-safe/-/uid-safe-2.1.5.tgz", + "integrity": "sha512-KPHm4VL5dDXKz01UuEd88Df+KzynaohSL9fBh096KWAxSKZQDI2uBrVqtvRM4rwrIrRRKsdLNML/lnaaVSRioA==", + "dependencies": { + "random-bytes": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/uid2": { + "version": "0.0.4", + "resolved": "https://registry.npmjs.org/uid2/-/uid2-0.0.4.tgz", + "integrity": "sha512-IevTus0SbGwQzYh3+fRsAMTVVPOoIVufzacXcHPmdlle1jUpq7BRL+mw3dgeLanvGZdwwbWhRV6XrcFNdBmjWA==" + }, + "node_modules/unc-path-regex": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/unc-path-regex/-/unc-path-regex-0.1.2.tgz", + "integrity": "sha512-eXL4nmJT7oCpkZsHZUOJo8hcX3GbsiDOa0Qu9F646fi8dT3XuSVopVqAcEiVzSKKH7UoDti23wNX3qGFxcW5Qg==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/undefsafe": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/undefsafe/-/undefsafe-2.0.5.tgz", + "integrity": "sha512-WxONCrssBM8TSPRqN5EmsjVrsv4A8X12J4ArBiiayv3DyyG3ZlIg6yysuuSYdZsVz3TKcTg2fd//Ujd4CHV1iA==", + "dev": true + }, + "node_modules/underscore": { + "version": "1.13.6", + "resolved": "https://registry.npmjs.org/underscore/-/underscore-1.13.6.tgz", + "integrity": "sha512-+A5Sja4HP1M08MaXya7p5LvjuM7K6q/2EaC0+iovj/wOcMsTzMvDFbasi/oSapiwOlt252IqsKqPjCl7huKS0A==", + "dev": true + }, + "node_modules/underscore.string": { + "version": "3.3.6", + "resolved": "https://registry.npmjs.org/underscore.string/-/underscore.string-3.3.6.tgz", + "integrity": "sha512-VoC83HWXmCrF6rgkyxS9GHv8W9Q5nhMKho+OadDJGzL2oDYbYEppBaCMH6pFlwLeqj2QS+hhkw2kpXkSdD1JxQ==", + "dev": true, + "dependencies": { + "sprintf-js": "^1.1.1", + "util-deprecate": "^1.0.2" + }, + "engines": { + "node": "*" + } + }, + "node_modules/underscore.string/node_modules/sprintf-js": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.1.2.tgz", + "integrity": "sha512-VE0SOVEHCk7Qc8ulkWw3ntAzXuqf7S2lvwQaDLRnUeIEaKNQJzV6BwmLKhOqT61aGhfUMrXeaBk+oDGCzvhcug==", + "dev": true + }, + "node_modules/unique-string": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/unique-string/-/unique-string-2.0.0.tgz", + "integrity": "sha512-uNaeirEPvpZWSgzwsPGtU2zVSTrn/8L5q/IexZmH0eH6SA73CmAA5U4GwORTxQAZs95TAXLNqeLoPPNO5gZfWg==", + "dev": true, + "dependencies": { + "crypto-random-string": "^2.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/universalify": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.0.tgz", + "integrity": "sha512-hAZsKq7Yy11Zu1DE0OzWjw7nnLZmJZYTDZZyEFHZdUhV8FkH5MCfoU1XMaxXovpyW5nq5scPqq0ZDP9Zyl04oQ==", + "engines": { + "node": ">= 10.0.0" + } + }, + "node_modules/unpipe": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz", + "integrity": "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/update-notifier": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/update-notifier/-/update-notifier-5.1.0.tgz", + "integrity": "sha512-ItnICHbeMh9GqUy31hFPrD1kcuZ3rpxDZbf4KUDavXwS0bW5m7SLbDQpGX3UYr072cbrF5hFUs3r5tUsPwjfHw==", + "dev": true, + "dependencies": { + "boxen": "^5.0.0", + "chalk": "^4.1.0", + "configstore": "^5.0.1", + "has-yarn": "^2.1.0", + "import-lazy": "^2.1.0", + "is-ci": "^2.0.0", + "is-installed-globally": "^0.4.0", + "is-npm": "^5.0.0", + "is-yarn-global": "^0.3.0", + "latest-version": "^5.1.0", + "pupa": "^2.1.1", + "semver": "^7.3.4", + "semver-diff": "^3.1.1", + "xdg-basedir": "^4.0.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/yeoman/update-notifier?sponsor=1" + } + }, + "node_modules/uri-js": { + "version": "4.4.1", + "resolved": "https://registry.npmjs.org/uri-js/-/uri-js-4.4.1.tgz", + "integrity": "sha512-7rKUyy33Q1yc98pQ1DAmLtwX109F7TIfWlW1Ydo8Wl1ii1SeHieeh0HHfPeL2fMXK6z0s8ecKs9frCuLJvndBg==", + "dependencies": { + "punycode": "^2.1.0" + } + }, + "node_modules/uri-path": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/uri-path/-/uri-path-1.0.0.tgz", + "integrity": "sha512-8pMuAn4KacYdGMkFaoQARicp4HSw24/DHOVKWqVRJ8LhhAwPPFpdGvdL9184JVmUwe7vz7Z9n6IqI6t5n2ELdg==", + "dev": true, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/url-parse-lax": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/url-parse-lax/-/url-parse-lax-3.0.0.tgz", + "integrity": "sha512-NjFKA0DidqPa5ciFcSrXnAltTtzz84ogy+NebPvfEgAck0+TNg4UJ4IN+fB7zRZfbgUf0syOo9MDxFkDSMuFaQ==", + "dev": true, + "dependencies": { + "prepend-http": "^2.0.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/util-deprecate": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", + "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==" + }, + "node_modules/utils-merge": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz", + "integrity": "sha512-pMZTvIkT1d+TFGvDOqodOclx0QWkkgi6Tdoa8gC8ffGAAqz9pzPTZWAybbsHHoED/ztMtkv/VoYTYyShUn81hA==", + "engines": { + "node": ">= 0.4.0" + } + }, + "node_modules/uuid": { + "version": "8.3.2", + "resolved": "https://registry.npmjs.org/uuid/-/uuid-8.3.2.tgz", + "integrity": "sha512-+NYs2QeMWy+GWFOEm9xnn6HCDp0l7QBD7ml8zLUmJ+93Q5NF0NocErnwkTkXVFNiX3/fpC6afS8Dhb/gz7R7eg==", + "bin": { + "uuid": "dist/bin/uuid" + } + }, + "node_modules/v8flags": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/v8flags/-/v8flags-3.2.0.tgz", + "integrity": "sha512-mH8etigqMfiGWdeXpaaqGfs6BndypxusHHcv2qSHyZkGEznCd/qAXCWWRzeowtL54147cktFOC4P5y+kl8d8Jg==", + "dev": true, + "dependencies": { + "homedir-polyfill": "^1.0.1" + }, + "engines": { + "node": ">= 0.10" + } + }, + "node_modules/validate-npm-package-license": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/validate-npm-package-license/-/validate-npm-package-license-3.0.4.tgz", + "integrity": "sha512-DpKm2Ui/xN7/HQKCtpZxoRWBhZ9Z0kqtygG8XCgNQ8ZlDnxuQmWhj566j8fN4Cu3/JmbhsDo7fcAJq4s9h27Ew==", + "dev": true, + "dependencies": { + "spdx-correct": "^3.0.0", + "spdx-expression-parse": "^3.0.0" + } + }, + "node_modules/vary": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz", + "integrity": "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==", + "engines": { + "node": ">= 0.8" + } + }, + "node_modules/walk-back": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/walk-back/-/walk-back-5.1.0.tgz", + "integrity": "sha512-Uhxps5yZcVNbLEAnb+xaEEMdgTXl9qAQDzKYejG2AZ7qPwRQ81lozY9ECDbjLPNWm7YsO1IK5rsP1KoQzXAcGA==", + "dev": true, + "engines": { + "node": ">=12.17" + } + }, + "node_modules/webidl-conversions": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz", + "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==", + "devOptional": true + }, + "node_modules/websocket-driver": { + "version": "0.7.4", + "resolved": "https://registry.npmjs.org/websocket-driver/-/websocket-driver-0.7.4.tgz", + "integrity": "sha512-b17KeDIQVjvb0ssuSDF2cYXSg2iztliJ4B9WdsuB6J952qCPKmnVq4DyW5motImXHDC1cBT/1UezrJVsKw5zjg==", + "dev": true, + "dependencies": { + "http-parser-js": ">=0.5.1", + "safe-buffer": ">=5.1.0", + "websocket-extensions": ">=0.1.1" + }, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/websocket-extensions": { + "version": "0.1.4", + "resolved": "https://registry.npmjs.org/websocket-extensions/-/websocket-extensions-0.1.4.tgz", + "integrity": "sha512-OqedPIGOfsDlo31UNwYbCFMSaO9m9G/0faIHj5/dZFDMFqPTcx6UwqyOy3COEaEOg/9VsGIpdqn62W5KhoKSpg==", + "dev": true, + "engines": { + "node": ">=0.8.0" + } + }, + "node_modules/whatwg-url": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz", + "integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==", + "devOptional": true, + "dependencies": { + "tr46": "~0.0.3", + "webidl-conversions": "^3.0.0" + } + }, + "node_modules/which": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz", + "integrity": "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==", + "dev": true, + "dependencies": { + "isexe": "^2.0.0" + }, + "bin": { + "node-which": "bin/node-which" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/which-module": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/which-module/-/which-module-2.0.0.tgz", + "integrity": "sha512-B+enWhmw6cjfVC7kS8Pj9pCrKSc5txArRyaYGe088shv/FGWH+0Rjx/xPgtsWfsUtS27FkP697E4DDhgrgoc0Q==", + "dev": true + }, + "node_modules/wide-align": { + "version": "1.1.5", + "resolved": "https://registry.npmjs.org/wide-align/-/wide-align-1.1.5.tgz", + "integrity": "sha512-eDMORYaPNZ4sQIuuYPDHdQvf4gyCF9rEEV/yPxGfwPkRodwEgiMUUXTx/dex+Me0wxx53S+NgUHaP7y3MGlDmg==", + "optional": true, + "dependencies": { + "string-width": "^1.0.2 || 2 || 3 || 4" + } + }, + "node_modules/widest-line": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/widest-line/-/widest-line-3.1.0.tgz", + "integrity": "sha512-NsmoXalsWVDMGupxZ5R08ka9flZjjiLvHVAWYOKtiKM8ujtZWr9cRffak+uSE48+Ob8ObalXpwyeUiyDD6QFgg==", + "dev": true, + "dependencies": { + "string-width": "^4.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/wordwrap": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-1.0.0.tgz", + "integrity": "sha512-gvVzJFlPycKc5dZN4yPkP8w7Dc37BtP1yczEneOb4uq34pXZcvrtRTmWV8W+Ume+XCxKgbjM+nevkyFPMybd4Q==", + "dev": true + }, + "node_modules/wordwrapjs": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/wordwrapjs/-/wordwrapjs-3.0.0.tgz", + "integrity": "sha512-mO8XtqyPvykVCsrwj5MlOVWvSnCdT+C+QVbm6blradR7JExAhbkZ7hZ9A+9NUtwzSqrlUo9a67ws0EiILrvRpw==", + "dev": true, + "dependencies": { + "reduce-flatten": "^1.0.1", + "typical": "^2.6.1" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/wordwrapjs/node_modules/reduce-flatten": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/reduce-flatten/-/reduce-flatten-1.0.1.tgz", + "integrity": "sha512-j5WfFJfc9CoXv/WbwVLHq74i/hdTUpy+iNC534LxczMRP67vJeK3V9JOdnL0N1cIRbn9mYhE2yVjvvKXDxvNXQ==", + "dev": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/workerpool": { + "version": "6.2.0", + "resolved": "https://registry.npmjs.org/workerpool/-/workerpool-6.2.0.tgz", + "integrity": "sha512-Rsk5qQHJ9eowMH28Jwhe8HEbmdYDX4lwoMWshiCXugjtHqMD9ZbiqSDLxcsfdqsETPzVUtX5s1Z5kStiIM6l4A==", + "dev": true + }, + "node_modules/wrap-ansi": { + "version": "7.0.0", + "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz", + "integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==", + "dev": true, + "dependencies": { + "ansi-styles": "^4.0.0", + "string-width": "^4.1.0", + "strip-ansi": "^6.0.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/chalk/wrap-ansi?sponsor=1" + } + }, + "node_modules/wrappy": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", + "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==" + }, + "node_modules/write-file-atomic": { + "version": "2.4.3", + "resolved": "https://registry.npmjs.org/write-file-atomic/-/write-file-atomic-2.4.3.tgz", + "integrity": "sha512-GaETH5wwsX+GcnzhPgKcKjJ6M2Cq3/iZp1WyY/X1CSqrW+jVNM9Y7D8EC2sM4ZG/V8wZlSniJnCKWPmBYAucRQ==", + "dev": true, + "dependencies": { + "graceful-fs": "^4.1.11", + "imurmurhash": "^0.1.4", + "signal-exit": "^3.0.2" + } + }, + "node_modules/ws": { + "version": "7.5.6", + "resolved": "https://registry.npmjs.org/ws/-/ws-7.5.6.tgz", + "integrity": "sha512-6GLgCqo2cy2A2rjCNFlxQS6ZljG/coZfZXclldI8FB/1G3CCI36Zd8xy2HrFVACi8tfk5XrgLQEk+P0Tnz9UcA==", + "engines": { + "node": ">=8.3.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": "^5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + }, + "node_modules/xdg-basedir": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/xdg-basedir/-/xdg-basedir-4.0.0.tgz", + "integrity": "sha512-PSNhEJDejZYV7h50BohL09Er9VaIefr2LMAf3OEmpCkjOi34eYyQYAXUTjEQtZJTKcF0E2UKTh+osDLsgNim9Q==", + "dev": true, + "engines": { + "node": ">=8" + } + }, + "node_modules/xml2js": { + "version": "0.4.23", + "resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.4.23.tgz", + "integrity": "sha512-ySPiMjM0+pLDftHgXY4By0uswI3SPKLDw/i3UXbnO8M/p28zqexCUoPmQFrYD+/1BzhGJSs2i1ERWKJAtiLrug==", + "dependencies": { + "sax": ">=0.6.0", + "xmlbuilder": "~11.0.0" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/xmlbuilder": { + "version": "11.0.1", + "resolved": "https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-11.0.1.tgz", + "integrity": "sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA==", + "engines": { + "node": ">=4.0" + } + }, + "node_modules/xmlcreate": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/xmlcreate/-/xmlcreate-2.0.4.tgz", + "integrity": "sha512-nquOebG4sngPmGPICTS5EnxqhKbCmz5Ox5hsszI2T6U5qdrJizBc+0ilYSEjTSzU0yZcmvppztXe/5Al5fUwdg==", + "dev": true + }, + "node_modules/xtend": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz", + "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==", + "engines": { + "node": ">=0.4" + } + }, + "node_modules/y18n": { + "version": "5.0.8", + "resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz", + "integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==", + "dev": true, + "engines": { + "node": ">=10" + } + }, + "node_modules/yallist": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-2.1.2.tgz", + "integrity": "sha512-ncTzHV7NvsQZkYe1DW7cbDLm0YpzHmZF5r/iyP3ZnQtMiJ+pjzisCiMNI+Sj+xQF5pXhSHxSB3uDbsBTzY/c2A==" + }, + "node_modules/yargs": { + "version": "16.2.0", + "resolved": "https://registry.npmjs.org/yargs/-/yargs-16.2.0.tgz", + "integrity": "sha512-D1mvvtDG0L5ft/jGWkLpG1+m0eQxOfaBvTNELraWj22wSVUMWxZUvYgJYcKh6jGGIkJFhH4IZPQhR4TKpc8mBw==", + "dev": true, + "dependencies": { + "cliui": "^7.0.2", + "escalade": "^3.1.1", + "get-caller-file": "^2.0.5", + "require-directory": "^2.1.1", + "string-width": "^4.2.0", + "y18n": "^5.0.5", + "yargs-parser": "^20.2.2" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/yargs-parser": { + "version": "20.2.4", + "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-20.2.4.tgz", + "integrity": "sha512-WOkpgNhPTlE73h4VFAFsOnomJVaovO8VqLDzy5saChRBFQFBoMYirowyW+Q9HB4HFF4Z7VZTiG3iSzJJA29yRA==", + "dev": true, + "engines": { + "node": ">=10" + } + }, + "node_modules/yargs-unparser": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/yargs-unparser/-/yargs-unparser-2.0.0.tgz", + "integrity": "sha512-7pRTIA9Qc1caZ0bZ6RYRGbHJthJWuakf+WmHK0rVeLkNrrGhfoabBNdue6kdINI6r4if7ocq9aD/n7xwKOdzOA==", + "dev": true, + "dependencies": { + "camelcase": "^6.0.0", + "decamelize": "^4.0.0", + "flat": "^5.0.2", + "is-plain-obj": "^2.1.0" + }, + "engines": { + "node": ">=10" + } + }, + "node_modules/yargs-unparser/node_modules/camelcase": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/camelcase/-/camelcase-6.3.0.tgz", + "integrity": "sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/yocto-queue": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/yocto-queue/-/yocto-queue-0.1.0.tgz", + "integrity": "sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q==", + "dev": true, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/zip-stream": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/zip-stream/-/zip-stream-4.1.0.tgz", + "integrity": "sha512-zshzwQW7gG7hjpBlgeQP9RuyPGNxvJdzR8SUM3QhxCnLjWN2E7j3dOvpeDcQoETfHx0urRS7EtmVToql7YpU4A==", + "dev": true, + "dependencies": { + "archiver-utils": "^2.1.0", + "compress-commons": "^4.1.0", + "readable-stream": "^3.6.0" + }, + "engines": { + "node": ">= 10" + } + } + } +} diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/package.json b/RAG-pipeline-with-nodered/RAG_with_node-red/example/package.json new file mode 100644 index 00000000..bd277eeb --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/package.json @@ -0,0 +1,127 @@ +{ + "name": "node-red", + "version": "2.2.3", + "description": "Low-code programming for event-driven applications", + "homepage": "http://nodered.org", + "license": "Apache-2.0", + "repository": { + "type": "git", + "url": "https://github.com/node-red/node-red.git" + }, + "private": "true", + "scripts": { + "start": "node $NODE_OPTIONS node_modules/node-red/red.js", + "test": "grunt", + "build": "grunt build", + "dev": "grunt dev", + "build-dev": "grunt build-dev", + "docs": "grunt docs" + }, + "contributors": [ + { + "name": "Nick O'Leary" + }, + { + "name": "Dave Conway-Jones" + } + ], + "dependencies": { + "acorn": "8.7.0", + "acorn-walk": "8.2.0", + "ajv": "8.10.0", + "async-mutex": "0.3.2", + "basic-auth": "2.0.1", + "bcryptjs": "2.4.3", + "body-parser": "1.19.1", + "cheerio": "1.0.0-rc.10", + "clone": "2.1.2", + "content-type": "1.0.4", + "cookie": "0.4.2", + "cookie-parser": "1.4.6", + "cors": "2.8.5", + "cronosjs": "1.7.1", + "denque": "2.0.1", + "express": "4.17.2", + "express-session": "1.17.2", + "form-data": "4.0.0", + "fs-extra": "10.0.0", + "fs.notify": "0.0.4", + "got": "11.8.3", + "hash-sum": "2.0.0", + "hpagent": "0.1.2", + "https-proxy-agent": "5.0.0", + "i18next": "21.6.11", + "iconv-lite": "0.6.3", + "is-utf8": "0.2.1", + "js-yaml": "3.14.1", + "json-stringify-safe": "5.0.1", + "jsonata": "1.8.6", + "lodash.clonedeep": "^4.5.0", + "media-typer": "1.1.0", + "memorystore": "1.6.7", + "mime": "3.0.0", + "moment-timezone": "0.5.34", + "mqtt": "4.3.5", + "multer": "1.4.4", + "mustache": "4.2.0", + "node-red": "2.2.3", + "node-red-admin": "^2.2.3", + "node-red-contrib-pythonshell": "github:namgk/node-red-contrib-pythonshell", + "nopt": "5.0.0", + "oauth2orize": "1.11.1", + "on-headers": "1.0.2", + "passport": "0.5.2", + "passport-http-bearer": "1.0.1", + "passport-oauth2-client-password": "0.1.2", + "raw-body": "2.4.3", + "semver": "7.3.5", + "tar": "6.1.11", + "tough-cookie": "4.0.0", + "uglify-js": "3.15.1", + "uuid": "8.3.2", + "ws": "7.5.6", + "xml2js": "0.4.23" + }, + "optionalDependencies": { + "bcrypt": "5.0.1" + }, + "devDependencies": { + "dompurify": "2.3.6", + "grunt": "1.5.2", + "grunt-chmod": "~1.1.1", + "grunt-cli": "~1.4.3", + "grunt-concurrent": "3.0.0", + "grunt-contrib-clean": "~2.0.0", + "grunt-contrib-compress": "2.0.0", + "grunt-contrib-concat": "~1.0.1", + "grunt-contrib-copy": "~1.0.0", + "grunt-contrib-jshint": "3.1.1", + "grunt-contrib-uglify": "5.0.1", + "grunt-contrib-watch": "~1.1.0", + "grunt-jsdoc": "2.4.1", + "grunt-jsdoc-to-markdown": "6.0.0", + "grunt-jsonlint": "2.1.3", + "grunt-mkdir": "~1.1.0", + "grunt-npm-command": "~0.1.2", + "grunt-sass": "~3.1.0", + "grunt-simple-mocha": "~0.4.1", + "grunt-simple-nyc": "^3.0.1", + "i18next-http-backend": "1.3.2", + "jquery-i18next": "1.2.1", + "jsdoc-nr-template": "github:node-red/jsdoc-nr-template", + "marked": "4.0.12", + "minami": "1.2.3", + "mocha": "9.2.0", + "node-red-node-test-helper": "^0.2.7", + "nodemon": "2.0.15", + "proxy": "^1.0.2", + "sass": "1.49.7", + "should": "13.2.3", + "sinon": "11.1.2", + "stoppable": "^1.1.0", + "supertest": "6.2.2" + }, + "engines": { + "node": ">=12" + } +} diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/run.sh b/RAG-pipeline-with-nodered/RAG_with_node-red/example/run.sh new file mode 100644 index 00000000..ec02fb96 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/run.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash + +target=$1 + +if [ "$#" -eq 1 ] +then + USERDIR=/data/$target + +fi + +echo "run with userdir=$USERDIR" + +USERDIR=$USERDIR docker compose up +#USERDIR=$USERDIR docker compose convert diff --git a/RAG-pipeline-with-nodered/RAG_with_node-red/example/scripts/entrypoint.sh b/RAG-pipeline-with-nodered/RAG_with_node-red/example/scripts/entrypoint.sh new file mode 100644 index 00000000..705054c0 --- /dev/null +++ b/RAG-pipeline-with-nodered/RAG_with_node-red/example/scripts/entrypoint.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +trap stop SIGINT SIGTERM + +function stop() { + kill $CHILD_PID + wait $CHILD_PID +} + +/usr/local/bin/node $NODE_OPTIONS node_modules/node-red/red.js --userDir $USERDIR & + +CHILD_PID="$!" + +wait "${CHILD_PID}" diff --git a/jpx-tokyo-stock-exchange-kaggle-competition/images/... b/RAG-pipeline-with-nodered/RAG_with_node-red/example/test similarity index 100% rename from jpx-tokyo-stock-exchange-kaggle-competition/images/... rename to RAG-pipeline-with-nodered/RAG_with_node-red/example/test diff --git a/RAG-pipeline-with-nodered/README.md b/RAG-pipeline-with-nodered/README.md new file mode 100644 index 00000000..9317744c --- /dev/null +++ b/RAG-pipeline-with-nodered/README.md @@ -0,0 +1,48 @@ +# RAG-on-kubeflow-with-nodered +A simple example of RAG on kubeflow with node-red + +## Implementation + +``` +git clone https://github.com/sefgsefg/RAG-pipeline-with-nodered.git +``` + +``` +cd RAG_with_node-red/example +``` + +``` +./run.sh main +``` + +Problem solve: -bash: ./run.sh: Permission denied +``` +chmod +x run.sh +``` + +``` +cd scripts +``` + +``` +chmod +x entrypoint.sh +``` + +``` +cd .. +``` +Run ./run.sh main again +``` +./run.sh main +``` +1.Insstall dependency and build the RAG flow on node-red. + +![](https://github.com/sefgsefg/RAG-pipeline-with-nodered/blob/main/node-red_1.png) + +2.Double click the RAG node and edit the infos. + +![](https://github.com/sefgsefg/RAG-pipeline-with-nodered/blob/main/node-red_2.png) + +3.Deploy and run the flow. It will run on the kubeflow pipeline. + +![](https://github.com/sefgsefg/RAG-pipeline-with-nodered/blob/main/pipeline.png) diff --git a/RAG-pipeline-with-nodered/node-red_1.png b/RAG-pipeline-with-nodered/node-red_1.png new file mode 100644 index 00000000..5b2f6f49 Binary files /dev/null and b/RAG-pipeline-with-nodered/node-red_1.png differ diff --git a/RAG-pipeline-with-nodered/node-red_2.png b/RAG-pipeline-with-nodered/node-red_2.png new file mode 100644 index 00000000..6a69afef Binary files /dev/null and b/RAG-pipeline-with-nodered/node-red_2.png differ diff --git a/RAG-pipeline-with-nodered/pipeline.png b/RAG-pipeline-with-nodered/pipeline.png new file mode 100644 index 00000000..0dfe54a5 Binary files /dev/null and b/RAG-pipeline-with-nodered/pipeline.png differ diff --git a/american-express-default-kaggle-competition/american-express-default-prediction-kale.ipynb b/american-express-default-kaggle-competition/american-express-default-prediction-kale.ipynb index 7f42edb3..6ac8e260 100644 --- a/american-express-default-kaggle-competition/american-express-default-prediction-kale.ipynb +++ b/american-express-default-kaggle-competition/american-express-default-prediction-kale.ipynb @@ -1,771 +1,771 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# 🪙 American Express - Default Prediction Competition Kale Pipeline\n", - "![](./images/background.jpg)\n", - "\n", - "---\n", - "\n", - "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to predict credit default. This competition is hosted by American Express. \n", - "\n", - "> American Express is a globally integrated payments company. The largest payment card issuer in the world, they provide customers with access to products, insights, and experiences that enrich lives and build business success.\n", - "\n", - "The dataset provided is an industrial scale data set of about 5.5 million rows. It has been pre-processed and converted to a lightweight version by raddar for ease of training and better result. This dataset is available in a [parquet format][1].\n", - "\n", - "[1]: https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Install necessary packages\n", - "\n", - "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", - "\n", - "NOTE: After installing python packages, restart notebook kernel before proceeding." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "!pip install -r requirements.txt --user --quiet" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Imports\n", - "\n", - "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "tags": [ - "imports" - ] - }, - "outputs": [], - "source": [ - "import numpy as np\n", - "import pandas as pd\n", - "import os, subprocess\n", - "import random, zipfile, joblib\n", - "import scipy.stats\n", - "import warnings\n", - "import gc, wget\n", - "\n", - "from sklearn.model_selection import StratifiedKFold\n", - "from lightgbm import LGBMClassifier\n", - "\n", - "warnings.filterwarnings(\"ignore\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Project hyper-parameters\n", - "\n", - "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "tags": [ - "pipeline-parameters" - ] - }, - "outputs": [], - "source": [ - "# Hyper-parameters\n", - "N_EST = 30\n", - "LR = 0.1" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "Set random seed for reproducibility" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "def fix_all_seeds(seed):\n", - " np.random.seed(seed)\n", - " random.seed(seed)\n", - " os.environ['PYTHONHASHSEED'] = str(seed)\n", - "\n", - "fix_all_seeds(2022)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Download data\n", - "\n", - "In this section, we download the data from kaggle using the Kaggle API credentials" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:download_data" - ] - }, - "outputs": [], - "source": [ - "# setup kaggle environment for data download\n", - "dataset = \"amex-data-integer-dtypes-parquet-format\"\n", - "\n", - "# setup kaggle environment for data download\n", - "with open('/secret/kaggle-secret/password', 'r') as file:\n", - " kaggle_key = file.read().rstrip()\n", - "with open('/secret/kaggle-secret/username', 'r') as file:\n", - " kaggle_user = file.read().rstrip()\n", - "\n", - "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", - "\n", - "# download kaggle's Amex-credit-prediction data\n", - "subprocess.run([\"kaggle\",\"datasets\", \"download\", \"-d\", f'raddar/{dataset}'])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:" - ] - }, - "outputs": [], - "source": [ - "# path to download to\n", - "data_path = 'data'\n", - "\n", - "# extract Amex-credit-prediction.zip to data_path\n", - "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", - " zip_ref.extractall(data_path)\n", - " \n", - "# download kaggle's Amex-credit-prediction train_labels.zip\n", - "download_link = \"https://github.com/kubeflow/examples/blob/master/american-express-default-kaggle-competition/data/train_labels.zip?raw=true\"\n", - "wget.download(download_link, f'{data_path}/train_labels.zip')\n", - "\n", - "# extract Amex-credit-prediction.zip to data_path\n", - "with zipfile.ZipFile(f'{data_path}/train_labels.zip','r') as zip_ref:\n", - " zip_ref.extractall(data_path)\n", - " \n", - "# delete zipfiles\n", - "subprocess.run(['rm', f'{dataset}.zip'])\n", - "subprocess.run(['rm', f'{data_path}/train_labels.zip'])" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Load the dataset\n", - "\n", - "First, let us load and analyze the data.\n", - "\n", - "The data is in csv format, thus, we use the handy read_csv pandas method." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "tags": [ - "block:load_data", - "prev:download_data" - ] - }, - "outputs": [], - "source": [ - "TRAIN_CSV = (f'{data_path}/train.parquet')\n", - "TEST_CSV = f'{data_path}/test.parquet'\n", - "TARGET_CSV = f'{data_path}/train_labels.csv'" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "target shape: (458913,)\n" - ] - } - ], - "source": [ - "df_train = pd.read_parquet(TRAIN_CSV)\n", - "df_test = pd.read_parquet(TEST_CSV)\n", - "target = pd.read_csv(TARGET_CSV).target.values\n", - "print(f\"target shape: {target.shape}\")" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "(5531451, 190)" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "customer_ID 0\n", - "S_2 0\n", - "P_2 45985\n", - "D_39 0\n", - "B_1 0\n", - " ... \n", - "D_141 101548\n", - "D_142 4587043\n", - "D_143 0\n", - "D_144 40727\n", - "D_145 0\n", - "Length: 190, dtype: int64" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.isna().sum()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "### Define Helper Functions" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "tags": [ - "functions" - ] - }, - "outputs": [], - "source": [ - "# @yunchonggan's fast metric implementation\n", - "# From https://www.kaggle.com/competitions/amex-default-prediction/discussion/328020\n", - "def amex_metric(y_true: np.array, y_pred: np.array) -> float:\n", - "\n", - " # count of positives and negatives\n", - " n_pos = y_true.sum()\n", - " n_neg = y_true.shape[0] - n_pos\n", - "\n", - " # sorting by descring prediction values\n", - " indices = np.argsort(y_pred)[::-1]\n", - " preds, target = y_pred[indices], y_true[indices]\n", - "\n", - " # filter the top 4% by cumulative row weights\n", - " weight = 20.0 - target * 19.0\n", - " cum_norm_weight = (weight / weight.sum()).cumsum()\n", - " four_pct_filter = cum_norm_weight <= 0.04\n", - "\n", - " # default rate captured at 4%\n", - " d = target[four_pct_filter].sum() / n_pos\n", - "\n", - " # weighted gini coefficient\n", - " lorentz = (target / n_pos).cumsum()\n", - " gini = ((lorentz - cum_norm_weight) * weight).sum()\n", - "\n", - " # max weighted gini coefficient\n", - " gini_max = 10 * n_neg * (1 - 19 / (n_pos + 20 * n_neg))\n", - "\n", - " # normalized weighted gini coefficient\n", - " g = gini / gini_max\n", - "\n", - " return 0.5 * (g + d)\n", - "\n", - "def lgb_amex_metric(y_true, y_pred):\n", - " \"\"\"The competition metric with lightgbm's calling convention\"\"\"\n", - " return ('amex_metric_score',\n", - " amex_metric(y_true, y_pred),\n", - " True)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Feature Engineering" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "tags": [ - "block:feature_engineering", - "prev:load_data" - ] - }, - "outputs": [], - "source": [ - "# feature engineering gotten from https://www.kaggle.com/code/ambrosm/amex-lightgbm-quickstart\n", - "def get_features(df, \n", - " features_avg, \n", - " features_min, \n", - " features_max, \n", - " features_last\n", - " ):\n", - " '''\n", - " This function takes a dataframe with all features and returns the aggregated feature grouped by the customer id.\n", - " \n", - " df - dataframe\n", - " '''\n", - " cid = pd.Categorical(df.pop('customer_ID'), ordered=True) # get customer id\n", - " last = (cid != np.roll(cid, -1)) # mask for last statement of every customer\n", - " \n", - " df_avg = (df\n", - " .groupby(cid)\n", - " .mean()[features_avg]\n", - " .rename(columns={f: f\"{f}_avg\" for f in features_avg})\n", - " ) \n", - " \n", - " df_min = (df\n", - " .groupby(cid)\n", - " .min()[features_min]\n", - " .rename(columns={f: f\"{f}_min\" for f in features_min})\n", - " )\n", - " gc.collect()\n", - " print('Computed min')\n", - " \n", - " df_max = (df\n", - " .groupby(cid)\n", - " .max()[features_max]\n", - " .rename(columns={f: f\"{f}_max\" for f in features_max})\n", - " )\n", - " gc.collect()\n", - " print('Computed max')\n", - " \n", - " df = (df.loc[last, features_last]\n", - " .rename(columns={f: f\"{f}_last\" for f in features_last})\n", - " .set_index(np.asarray(cid[last]))\n", - " )\n", - " gc.collect()\n", - " print('Computed last')\n", - " \n", - " df_ = pd.concat([df, df_min, df_max, df_avg], axis=1, )\n", - " \n", - " del df, df_avg, df_min, df_max, cid, last\n", - " \n", - " return df_" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "features_avg = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', \n", - " 'B_16', 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_28', 'B_29', 'B_30', \n", - " 'B_32', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', \n", - " 'D_45', 'D_46', 'D_47', 'D_48', 'D_50', 'D_51', 'D_53', 'D_54', 'D_55', 'D_58', 'D_59', 'D_60', 'D_61', \n", - " 'D_62', 'D_65', 'D_66', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_75', 'D_76', 'D_77', 'D_78', \n", - " 'D_80', 'D_82', 'D_84', 'D_86', 'D_91', 'D_92', 'D_94', 'D_96', 'D_103', 'D_104', 'D_108', 'D_112', 'D_113', \n", - " 'D_114', 'D_115', 'D_117', 'D_118', 'D_119', 'D_120', 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', \n", - " 'D_128', 'D_129', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', 'D_140', 'D_141', 'D_142', 'D_144', \n", - " 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_2', 'R_3', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_14', 'R_15', 'R_16', \n", - " 'R_17', 'R_20', 'R_21', 'R_22', 'R_24', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_9', 'S_11', 'S_12', 'S_13', \n", - " 'S_15', 'S_16', 'S_18', 'S_22', 'S_23', 'S_25', 'S_26']\n", - "features_min = ['B_2', 'B_4', 'B_5', 'B_9', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', 'B_19', 'B_20', 'B_28', 'B_29', 'B_33', 'B_36', \n", - " 'B_42', 'D_39', 'D_41', 'D_42', 'D_45', 'D_46', 'D_48', 'D_50', 'D_51', 'D_53', 'D_55', 'D_56', 'D_58', 'D_59', \n", - " 'D_60', 'D_62', 'D_70', 'D_71', 'D_74', 'D_75', 'D_78', 'D_83', 'D_102', 'D_112', 'D_113', 'D_115', 'D_118', 'D_119', \n", - " 'D_121', 'D_122', 'D_128', 'D_132', 'D_140', 'D_141', 'D_144', 'D_145', 'P_2', 'P_3', 'R_1', 'R_27', 'S_3', 'S_5', \n", - " 'S_7', 'S_9', 'S_11', 'S_12', 'S_23', 'S_25']\n", - "features_max = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', \n", - " 'B_18', 'B_19', 'B_21', 'B_23', 'B_24', 'B_25', 'B_29', 'B_30', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_42', 'D_39', \n", - " 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', 'D_48', 'D_49', 'D_50', 'D_52', 'D_55', 'D_56', 'D_58', 'D_59', \n", - " 'D_60', 'D_61', 'D_63', 'D_64', 'D_65', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_76', 'D_77', 'D_78', 'D_80', 'D_82', \n", - " 'D_84', 'D_91', 'D_102', 'D_105', 'D_107', 'D_110', 'D_111', 'D_112', 'D_115', 'D_116', 'D_117', 'D_118', 'D_119', \n", - " 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', 'D_128', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', \n", - " 'D_138', 'D_140', 'D_141', 'D_142', 'D_144', 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_3', 'R_5', 'R_6', 'R_7', 'R_8', \n", - " 'R_10', 'R_11', 'R_14', 'R_17', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_7', 'S_8', 'S_11', 'S_12', 'S_13', 'S_15', 'S_16', \n", - " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", - "features_last = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', \n", - " 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_26', 'B_28', 'B_29', 'B_30', 'B_32', 'B_33', \n", - " 'B_36', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', \n", - " 'D_48', 'D_49', 'D_50', 'D_51', 'D_52', 'D_53', 'D_54', 'D_55', 'D_56', 'D_58', 'D_59', 'D_60', 'D_61', 'D_62', 'D_63', \n", - " 'D_64', 'D_65', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_75', 'D_76', 'D_77', 'D_78', 'D_79', 'D_80', 'D_81', 'D_82', \n", - " 'D_83', 'D_86', 'D_91', 'D_96', 'D_105', 'D_106', 'D_112', 'D_114', 'D_119', 'D_120', 'D_121', 'D_122', 'D_124', 'D_125', \n", - " 'D_126', 'D_127', 'D_130', 'D_131', 'D_132', 'D_133', 'D_134', 'D_138', 'D_140', 'D_141', 'D_142', 'D_145', 'P_2', 'P_3', \n", - " 'P_4', 'R_1', 'R_2', 'R_3', 'R_4', 'R_5', 'R_6', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_12', 'R_13', 'R_14', 'R_15', \n", - " 'R_19', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_8', 'S_9', 'S_11', 'S_12', 'S_13', 'S_16', 'S_19', 'S_20', \n", - " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# apply feature engineering function\n", - "train = get_features(df_train, features_avg, features_min, features_max, features_last)\n", - "test = get_features(df_test, features_avg, features_min, features_max, features_last)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# check null values\n", - "train.isna().any()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Modelling: StratifiedKFold\n", - "\n", - "We cross-validate with a six-fold StratifiedKFold to handle the imbalanced nature of the target.\n", - "\n", - "Lightgbm handles null values efficiently." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:modelling", - "prev:feature_engineering" - ] - }, - "outputs": [], - "source": [ - "# Cross-validation\n", - "\n", - "features = [f for f in train.columns if f != 'customer_ID' and f != 'target']\n", - "\n", - "print(f\"{len(features)} features\")\n", - "\n", - "score_list = [] # lgbm score per fold\n", - "y_pred_list = [] # fold predictions list\n", - "\n", - "# init StratifiedKFold\n", - "kf = StratifiedKFold(n_splits=4)\n", - "\n", - "for fold, (idx_tr, idx_va) in enumerate(kf.split(train, target)):\n", - " \n", - " X_tr, X_va, y_tr, y_va, model = None, None, None, None, None\n", - "\n", - " X_tr = train.iloc[idx_tr][features]\n", - " X_va = train.iloc[idx_va][features]\n", - " y_tr = target[idx_tr]\n", - " y_va = target[idx_va]\n", - " \n", - " # init model\n", - " model = LGBMClassifier(n_estimators=int(N_EST),\n", - " learning_rate=float(LR), \n", - " random_state=2022)\n", - " # fit model\n", - " model.fit(X_tr, y_tr,\n", - " eval_set = [(X_va, y_va)], \n", - " eval_metric=[lgb_amex_metric],\n", - " verbose = 20,\n", - " early_stopping_rounds=30)\n", - " \n", - " X_tr, y_tr = None, None\n", - " \n", - " # fold validation set predictions\n", - " y_va_pred = model.predict_proba(X_va, raw_score=True)\n", - " \n", - " # model score\n", - " score = amex_metric(y_va, y_va_pred)\n", - "\n", - " print(f\"Score = {score}\")\n", - " score_list.append(score)\n", - " \n", - " # test set predictions\n", - " y_pred_list.append(model.predict_proba(test[features], raw_score=True))\n", - " \n", - " print(f\"Fold {fold}\") \n", - "\n", - "# save model\n", - "joblib.dump(model, 'lgb.jl')\n", - "print(f\"OOF Score: {np.mean(score_list):.5f}\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# feature importance for top 30 features\n", - "fea_imp = pd.DataFrame({'imp':model.feature_importances_, 'col': features})\n", - "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", - "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Evaluation" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "block:evaluation_result", - "prev:modelling" - ] - }, - "outputs": [], - "source": [ - "model = joblib.load('lgb.jl')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "binary_logloss = model.booster_.best_score.get('valid_0').get('binary_logloss')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "amex_metric_score = model.booster_.best_score.get('valid_0').get('amex_metric_score')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [], - "source": [ - "print(binary_logloss)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [], - "source": [ - "print(amex_metric_score)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Submission" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "sub = pd.DataFrame({'customer_ID': test.index,\n", - " 'prediction': np.mean(y_pred_list, axis=0)})\n", - "sub.to_csv('submission.csv', index=False)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "sub" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "kubeflow_notebook": { - "autosnapshot": true, - "experiment": { - "id": "new", - "name": "american-express-defaul-prediction" - }, - "experiment_name": "american-express-defaul-prediction", - "katib_metadata": { - "algorithm": { - "algorithmName": "grid" - }, - "maxFailedTrialCount": 3, - "maxTrialCount": 12, - "objective": { - "objectiveMetricName": "", - "type": "minimize" - }, - "parallelTrialCount": 3, - "parameters": [] - }, - "katib_run": false, - "pipeline_description": "predicting credit default", - "pipeline_name": "american-express-defaul-prediction-pipeline", - "snapshot_volumes": true, - "steps_defaults": [ - "label:access-ml-pipeline:true", - "label:kaggle-secret:true", - "label:access-rok:true" - ], - "volume_access_mode": "rwm", - "volumes": [ - { - "annotations": [], - "mount_point": "/home/jovyan", - "name": "test-workspace-qtvmt", - "size": 32, - "size_type": "Gi", - "snapshot": false, - "type": "clone" - } - ] - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# 🪙 American Express - Default Prediction Competition Kale Pipeline\n", + "![](./images/background.jpg)\n", + "\n", + "---\n", + "\n", + "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to predict credit default. This competition is hosted by American Express. \n", + "\n", + "> American Express is a globally integrated payments company. The largest payment card issuer in the world, they provide customers with access to products, insights, and experiences that enrich lives and build business success.\n", + "\n", + "The dataset provided is an industrial scale data set of about 5.5 million rows. It has been pre-processed and converted to a lightweight version by raddar for ease of training and better result. This dataset is available in a [parquet format][1].\n", + "\n", + "[1]: https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Install necessary packages\n", + "\n", + "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", + "\n", + "NOTE: After installing python packages, restart notebook kernel before proceeding." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "!pip install -r requirements.txt --user --quiet" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Imports\n", + "\n", + "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "tags": [ + "imports" + ] + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import os, subprocess\n", + "import random, zipfile, joblib\n", + "import scipy.stats\n", + "import warnings\n", + "import gc, wget\n", + "\n", + "from sklearn.model_selection import StratifiedKFold\n", + "from lightgbm import LGBMClassifier\n", + "\n", + "warnings.filterwarnings(\"ignore\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Project hyper-parameters\n", + "\n", + "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "pipeline-parameters" + ] + }, + "outputs": [], + "source": [ + "# Hyper-parameters\n", + "N_EST = 30\n", + "LR = 0.1" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "Set random seed for reproducibility" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "def fix_all_seeds(seed):\n", + " np.random.seed(seed)\n", + " random.seed(seed)\n", + " os.environ['PYTHONHASHSEED'] = str(seed)\n", + "\n", + "fix_all_seeds(2022)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Download data\n", + "\n", + "In this section, we download the data from kaggle using the Kaggle API credentials" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:download_data" + ] + }, + "outputs": [], + "source": [ + "# setup kaggle environment for data download\n", + "dataset = \"amex-data-integer-dtypes-parquet-format\"\n", + "\n", + "# setup kaggle environment for data download\n", + "with open('/secret/kaggle-secret/password', 'r') as file:\n", + " kaggle_key = file.read().rstrip()\n", + "with open('/secret/kaggle-secret/username', 'r') as file:\n", + " kaggle_user = file.read().rstrip()\n", + "\n", + "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", + "\n", + "# download kaggle's Amex-credit-prediction data\n", + "subprocess.run([\"kaggle\",\"datasets\", \"download\", \"-d\", f'raddar/{dataset}'])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:" + ] + }, + "outputs": [], + "source": [ + "# path to download to\n", + "data_path = 'data'\n", + "\n", + "# extract Amex-credit-prediction.zip to data_path\n", + "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", + " zip_ref.extractall(data_path)\n", + " \n", + "# download kaggle's Amex-credit-prediction train_labels.zip\n", + "download_link = \"https://github.com/kubeflow/examples/blob/master/american-express-default-kaggle-competition/data/train_labels.zip?raw=true\"\n", + "wget.download(download_link, f'{data_path}/train_labels.zip')\n", + "\n", + "# extract Amex-credit-prediction.zip to data_path\n", + "with zipfile.ZipFile(f'{data_path}/train_labels.zip','r') as zip_ref:\n", + " zip_ref.extractall(data_path)\n", + " \n", + "# delete zipfiles\n", + "subprocess.run(['rm', f'{dataset}.zip'])\n", + "subprocess.run(['rm', f'{data_path}/train_labels.zip'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Load the dataset\n", + "\n", + "First, let us load and analyze the data.\n", + "\n", + "The data is in csv format, thus, we use the handy read_csv pandas method." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "tags": [ + "block:load_data", + "prev:download_data" + ] + }, + "outputs": [], + "source": [ + "TRAIN_CSV = (f'{data_path}/train.parquet')\n", + "TEST_CSV = f'{data_path}/test.parquet'\n", + "TARGET_CSV = f'{data_path}/train_labels.csv'" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "target shape: (458913,)\n" + ] + } + ], + "source": [ + "df_train = pd.read_parquet(TRAIN_CSV)\n", + "df_test = pd.read_parquet(TEST_CSV)\n", + "target = pd.read_csv(TARGET_CSV).target.values\n", + "print(f\"target shape: {target.shape}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(5531451, 190)" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "customer_ID 0\n", + "S_2 0\n", + "P_2 45985\n", + "D_39 0\n", + "B_1 0\n", + " ... \n", + "D_141 101548\n", + "D_142 4587043\n", + "D_143 0\n", + "D_144 40727\n", + "D_145 0\n", + "Length: 190, dtype: int64" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.isna().sum()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "### Define Helper Functions" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "tags": [ + "functions" + ] + }, + "outputs": [], + "source": [ + "# @yunchonggan's fast metric implementation\n", + "# From https://www.kaggle.com/competitions/amex-default-prediction/discussion/328020\n", + "def amex_metric(y_true: np.array, y_pred: np.array) -> float:\n", + "\n", + " # count of positives and negatives\n", + " n_pos = y_true.sum()\n", + " n_neg = y_true.shape[0] - n_pos\n", + "\n", + " # sorting by descring prediction values\n", + " indices = np.argsort(y_pred)[::-1]\n", + " preds, target = y_pred[indices], y_true[indices]\n", + "\n", + " # filter the top 4% by cumulative row weights\n", + " weight = 20.0 - target * 19.0\n", + " cum_norm_weight = (weight / weight.sum()).cumsum()\n", + " four_pct_filter = cum_norm_weight <= 0.04\n", + "\n", + " # default rate captured at 4%\n", + " d = target[four_pct_filter].sum() / n_pos\n", + "\n", + " # weighted gini coefficient\n", + " lorentz = (target / n_pos).cumsum()\n", + " gini = ((lorentz - cum_norm_weight) * weight).sum()\n", + "\n", + " # max weighted gini coefficient\n", + " gini_max = 10 * n_neg * (1 - 19 / (n_pos + 20 * n_neg))\n", + "\n", + " # normalized weighted gini coefficient\n", + " g = gini / gini_max\n", + "\n", + " return 0.5 * (g + d)\n", + "\n", + "def lgb_amex_metric(y_true, y_pred):\n", + " \"\"\"The competition metric with lightgbm's calling convention\"\"\"\n", + " return ('amex_metric_score',\n", + " amex_metric(y_true, y_pred),\n", + " True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Feature Engineering" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "tags": [ + "block:feature_engineering", + "prev:load_data" + ] + }, + "outputs": [], + "source": [ + "# feature engineering gotten from https://www.kaggle.com/code/ambrosm/amex-lightgbm-quickstart\n", + "def get_features(df, \n", + " features_avg, \n", + " features_min, \n", + " features_max, \n", + " features_last\n", + " ):\n", + " '''\n", + " This function takes a dataframe with all features and returns the aggregated feature grouped by the customer id.\n", + " \n", + " df - dataframe\n", + " '''\n", + " cid = pd.Categorical(df.pop('customer_ID'), ordered=True) # get customer id\n", + " last = (cid != np.roll(cid, -1)) # mask for last statement of every customer\n", + " \n", + " df_avg = (df\n", + " .groupby(cid)\n", + " .mean()[features_avg]\n", + " .rename(columns={f: f\"{f}_avg\" for f in features_avg})\n", + " ) \n", + " \n", + " df_min = (df\n", + " .groupby(cid)\n", + " .min()[features_min]\n", + " .rename(columns={f: f\"{f}_min\" for f in features_min})\n", + " )\n", + " gc.collect()\n", + " print('Computed min')\n", + " \n", + " df_max = (df\n", + " .groupby(cid)\n", + " .max()[features_max]\n", + " .rename(columns={f: f\"{f}_max\" for f in features_max})\n", + " )\n", + " gc.collect()\n", + " print('Computed max')\n", + " \n", + " df = (df.loc[last, features_last]\n", + " .rename(columns={f: f\"{f}_last\" for f in features_last})\n", + " .set_index(np.asarray(cid[last]))\n", + " )\n", + " gc.collect()\n", + " print('Computed last')\n", + " \n", + " df_ = pd.concat([df, df_min, df_max, df_avg], axis=1, )\n", + " \n", + " del df, df_avg, df_min, df_max, cid, last\n", + " \n", + " return df_" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "features_avg = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', \n", + " 'B_16', 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_28', 'B_29', 'B_30', \n", + " 'B_32', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', \n", + " 'D_45', 'D_46', 'D_47', 'D_48', 'D_50', 'D_51', 'D_53', 'D_54', 'D_55', 'D_58', 'D_59', 'D_60', 'D_61', \n", + " 'D_62', 'D_65', 'D_66', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_75', 'D_76', 'D_77', 'D_78', \n", + " 'D_80', 'D_82', 'D_84', 'D_86', 'D_91', 'D_92', 'D_94', 'D_96', 'D_103', 'D_104', 'D_108', 'D_112', 'D_113', \n", + " 'D_114', 'D_115', 'D_117', 'D_118', 'D_119', 'D_120', 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', \n", + " 'D_128', 'D_129', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', 'D_140', 'D_141', 'D_142', 'D_144', \n", + " 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_2', 'R_3', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_14', 'R_15', 'R_16', \n", + " 'R_17', 'R_20', 'R_21', 'R_22', 'R_24', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_9', 'S_11', 'S_12', 'S_13', \n", + " 'S_15', 'S_16', 'S_18', 'S_22', 'S_23', 'S_25', 'S_26']\n", + "features_min = ['B_2', 'B_4', 'B_5', 'B_9', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', 'B_19', 'B_20', 'B_28', 'B_29', 'B_33', 'B_36', \n", + " 'B_42', 'D_39', 'D_41', 'D_42', 'D_45', 'D_46', 'D_48', 'D_50', 'D_51', 'D_53', 'D_55', 'D_56', 'D_58', 'D_59', \n", + " 'D_60', 'D_62', 'D_70', 'D_71', 'D_74', 'D_75', 'D_78', 'D_83', 'D_102', 'D_112', 'D_113', 'D_115', 'D_118', 'D_119', \n", + " 'D_121', 'D_122', 'D_128', 'D_132', 'D_140', 'D_141', 'D_144', 'D_145', 'P_2', 'P_3', 'R_1', 'R_27', 'S_3', 'S_5', \n", + " 'S_7', 'S_9', 'S_11', 'S_12', 'S_23', 'S_25']\n", + "features_max = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', \n", + " 'B_18', 'B_19', 'B_21', 'B_23', 'B_24', 'B_25', 'B_29', 'B_30', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_42', 'D_39', \n", + " 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', 'D_48', 'D_49', 'D_50', 'D_52', 'D_55', 'D_56', 'D_58', 'D_59', \n", + " 'D_60', 'D_61', 'D_63', 'D_64', 'D_65', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_76', 'D_77', 'D_78', 'D_80', 'D_82', \n", + " 'D_84', 'D_91', 'D_102', 'D_105', 'D_107', 'D_110', 'D_111', 'D_112', 'D_115', 'D_116', 'D_117', 'D_118', 'D_119', \n", + " 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', 'D_128', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', \n", + " 'D_138', 'D_140', 'D_141', 'D_142', 'D_144', 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_3', 'R_5', 'R_6', 'R_7', 'R_8', \n", + " 'R_10', 'R_11', 'R_14', 'R_17', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_7', 'S_8', 'S_11', 'S_12', 'S_13', 'S_15', 'S_16', \n", + " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", + "features_last = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', \n", + " 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_26', 'B_28', 'B_29', 'B_30', 'B_32', 'B_33', \n", + " 'B_36', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', \n", + " 'D_48', 'D_49', 'D_50', 'D_51', 'D_52', 'D_53', 'D_54', 'D_55', 'D_56', 'D_58', 'D_59', 'D_60', 'D_61', 'D_62', 'D_63', \n", + " 'D_64', 'D_65', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_75', 'D_76', 'D_77', 'D_78', 'D_79', 'D_80', 'D_81', 'D_82', \n", + " 'D_83', 'D_86', 'D_91', 'D_96', 'D_105', 'D_106', 'D_112', 'D_114', 'D_119', 'D_120', 'D_121', 'D_122', 'D_124', 'D_125', \n", + " 'D_126', 'D_127', 'D_130', 'D_131', 'D_132', 'D_133', 'D_134', 'D_138', 'D_140', 'D_141', 'D_142', 'D_145', 'P_2', 'P_3', \n", + " 'P_4', 'R_1', 'R_2', 'R_3', 'R_4', 'R_5', 'R_6', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_12', 'R_13', 'R_14', 'R_15', \n", + " 'R_19', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_8', 'S_9', 'S_11', 'S_12', 'S_13', 'S_16', 'S_19', 'S_20', \n", + " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# apply feature engineering function\n", + "train = get_features(df_train, features_avg, features_min, features_max, features_last)\n", + "test = get_features(df_test, features_avg, features_min, features_max, features_last)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# check null values\n", + "train.isna().any()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Modelling: StratifiedKFold\n", + "\n", + "We cross-validate with a six-fold StratifiedKFold to handle the imbalanced nature of the target.\n", + "\n", + "Lightgbm handles null values efficiently." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:modelling", + "prev:feature_engineering" + ] + }, + "outputs": [], + "source": [ + "# Cross-validation\n", + "\n", + "features = [f for f in train.columns if f != 'customer_ID' and f != 'target']\n", + "\n", + "print(f\"{len(features)} features\")\n", + "\n", + "score_list = [] # lgbm score per fold\n", + "y_pred_list = [] # fold predictions list\n", + "\n", + "# init StratifiedKFold\n", + "kf = StratifiedKFold(n_splits=4)\n", + "\n", + "for fold, (idx_tr, idx_va) in enumerate(kf.split(train, target)):\n", + " \n", + " X_tr, X_va, y_tr, y_va, model = None, None, None, None, None\n", + "\n", + " X_tr = train.iloc[idx_tr][features]\n", + " X_va = train.iloc[idx_va][features]\n", + " y_tr = target[idx_tr]\n", + " y_va = target[idx_va]\n", + " \n", + " # init model\n", + " model = LGBMClassifier(n_estimators=int(N_EST),\n", + " learning_rate=float(LR), \n", + " random_state=2022)\n", + " # fit model\n", + " model.fit(X_tr, y_tr,\n", + " eval_set = [(X_va, y_va)], \n", + " eval_metric=[lgb_amex_metric],\n", + " verbose = 20,\n", + " early_stopping_rounds=30)\n", + " \n", + " X_tr, y_tr = None, None\n", + " \n", + " # fold validation set predictions\n", + " y_va_pred = model.predict_proba(X_va, raw_score=True)\n", + " \n", + " # model score\n", + " score = amex_metric(y_va, y_va_pred)\n", + "\n", + " print(f\"Score = {score}\")\n", + " score_list.append(score)\n", + " \n", + " # test set predictions\n", + " y_pred_list.append(model.predict_proba(test[features], raw_score=True))\n", + " \n", + " print(f\"Fold {fold}\") \n", + "\n", + "# save model\n", + "joblib.dump(model, 'lgb.jl')\n", + "print(f\"OOF Score: {np.mean(score_list):.5f}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# feature importance for top 30 features\n", + "fea_imp = pd.DataFrame({'imp':model.feature_importances_, 'col': features})\n", + "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", + "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Evaluation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "block:evaluation_result", + "prev:modelling" + ] + }, + "outputs": [], + "source": [ + "model = joblib.load('lgb.jl')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "binary_logloss = model.booster_.best_score.get('valid_0').get('binary_logloss')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "amex_metric_score = model.booster_.best_score.get('valid_0').get('amex_metric_score')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [], + "source": [ + "print(binary_logloss)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [], + "source": [ + "print(amex_metric_score)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Submission" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "sub = pd.DataFrame({'customer_ID': test.index,\n", + " 'prediction': np.mean(y_pred_list, axis=0)})\n", + "sub.to_csv('submission.csv', index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "sub" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "kubeflow_notebook": { + "autosnapshot": true, + "experiment": { + "id": "new", + "name": "american-express-defaul-prediction" + }, + "experiment_name": "american-express-defaul-prediction", + "katib_metadata": { + "algorithm": { + "algorithmName": "grid" + }, + "maxFailedTrialCount": 3, + "maxTrialCount": 12, + "objective": { + "objectiveMetricName": "", + "type": "minimize" + }, + "parallelTrialCount": 3, + "parameters": [] + }, + "katib_run": false, + "pipeline_description": "predicting credit default", + "pipeline_name": "american-express-defaul-prediction-pipeline", + "snapshot_volumes": true, + "steps_defaults": [ + "label:access-ml-pipeline:true", + "label:kaggle-secret:true", + "label:access-rok:true" + ], + "volume_access_mode": "rwm", + "volumes": [ + { + "annotations": [], + "mount_point": "/home/jovyan", + "name": "test-workspace-qtvmt", + "size": 32, + "size_type": "Gi", + "snapshot": false, + "type": "clone" + } + ] + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/american-express-default-kaggle-competition/american-express-default-prediction-kfp.ipynb b/american-express-default-kaggle-competition/american-express-default-prediction-kfp.ipynb index 01ffdac4..6f585c8c 100644 --- a/american-express-default-kaggle-competition/american-express-default-prediction-kfp.ipynb +++ b/american-express-default-kaggle-competition/american-express-default-prediction-kfp.ipynb @@ -1,779 +1,779 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# 🪙 American Express - Default Prediction Competition Vanilla KFP Pipeline\n", - "![](./images/background.jpg)\n", - "\n", - "---\n", - "\n", - "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to predict credit default. This competition is hosted by American Express. \n", - "\n", - "> American Express is a globally integrated payments company. The largest payment card issuer in the world, they provide customers with access to products, insights, and experiences that enrich lives and build business success.\n", - "\n", - "The dataset provided is an industrial scale data set of about 5.5 million rows. It has been pre-processed and converted to a lightweight version by raddar for ease of training and better result. This dataset is available in a [parquet format][1].\n", - "\n", - "[1]: https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Install relevant libraries\n", - "\n", - "\n", - ">Update pip `pip install --user --upgrade pip`\n", - "\n", - ">Install and upgrade kubeflow sdk `pip install kfp --upgrade --user --quiet`\n", - "\n", - "You may need to restart your notebook kernel after installing the kfp sdk" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Requirement already satisfied: pip in /usr/local/lib/python3.6/dist-packages (21.3.1)\n" - ] - } - ], - "source": [ - "!pip install --user --upgrade pip" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "!pip install kfp --upgrade --user --quiet" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Name: kfp\n", - "Version: 1.8.11\n", - "Summary: KubeFlow Pipelines SDK\n", - "Home-page: https://github.com/kubeflow/pipelines\n", - "Author: The Kubeflow Authors\n", - "Author-email: \n", - "License: UNKNOWN\n", - "Location: /home/jovyan/.local/lib/python3.6/site-packages\n", - "Requires: absl-py, click, cloudpickle, dataclasses, Deprecated, docstring-parser, fire, google-api-python-client, google-auth, google-cloud-storage, jsonschema, kfp-pipeline-spec, kfp-server-api, kubernetes, protobuf, pydantic, PyYAML, requests-toolbelt, strip-hints, tabulate, typer, typing-extensions, uritemplate\n", - "Required-by: kubeflow-kale\n" - ] - } - ], - "source": [ - "# confirm the kfp sdk\n", - "! pip show kfp" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "tags": [ - "imports" - ] - }, - "outputs": [], - "source": [ - "import kfp\n", - "import kfp.components as comp\n", - "import kfp.dsl as dsl\n", - "from kfp.components import OutputPath\n", - "from typing import NamedTuple" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# Kubeflow pipeline component creation\n", - "\n", - "## Download the dataset" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [], - "source": [ - "# load data step\n", - "def download_data(dataset, \n", - " data_path):\n", - " \n", - " # install the necessary libraries\n", - " import os, sys, subprocess, zipfile, pickle;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','kaggle'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','wget'])\n", - " \n", - " # import libraries\n", - " import pandas as pd\n", - " import wget\n", - "\n", - " # setup kaggle environment for data download\n", - " with open('/secret/kaggle-secret/password', 'r') as file:\n", - " kaggle_key = file.read().rstrip()\n", - " with open('/secret/kaggle-secret/username', 'r') as file:\n", - " kaggle_user = file.read().rstrip()\n", - " \n", - " os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", - " \n", - " # create data_path directory\n", - " if not os.path.exists(data_path):\n", - " os.makedirs(data_path)\n", - " \n", - " # download kaggle's Amex-credit-prediction data\n", - " subprocess.run([\"kaggle\",\"datasets\", \"download\", \"-d\", f'raddar/{dataset}'])\n", - " \n", - " # extract Amex-credit-prediction.zip to data_path\n", - " with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", - " zip_ref.extractall(data_path)\n", - "\n", - " # download kaggle's Amex-credit-prediction train_labels.zip\n", - " download_link = \"https://github.com/kubeflow/examples/blob/master/american-express-default-kaggle-competition/data/train_labels.zip?raw=true\"\n", - " \n", - " wget.download(download_link, f'{data_path}/train_labels.zip')\n", - "\n", - " # extract Amex-credit-prediction.zip to data_path\n", - " with zipfile.ZipFile(f'{data_path}/train_labels.zip','r') as zip_ref:\n", - " zip_ref.extractall(data_path)\n", - "\n", - " # delete zipfiles\n", - " subprocess.run(['rm', f'{dataset}.zip'])\n", - " subprocess.run(['rm', f'{data_path}/train_labels.zip'])\n", - " return(print('Done!'))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Load Data" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [], - "source": [ - "# load data step\n", - "def load_data(data_path):\n", - " \n", - " # install the necessary libraries\n", - " import os, sys, subprocess, pickle;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pyarrow'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','fastparquet'])\n", - " \n", - " # import libraries\n", - " import pandas as pd\n", - "\n", - " TRAIN_CSV = (f'{data_path}/train.parquet')\n", - " TEST_CSV = f'{data_path}/test.parquet'\n", - " TARGET_CSV = f'{data_path}/train_labels.csv'\n", - " \n", - " # read parquet TRAIN, TEST and TARGET_CSV\n", - " df_train = pd.read_parquet(TRAIN_CSV)\n", - " df_test = pd.read_parquet(TEST_CSV)\n", - " target = pd.read_csv(TARGET_CSV).target.values\n", - " print(f\"target shape: {target.shape}\")\n", - " \n", - " \n", - " # Save all data as a pickle file to be used by the feature_engineering component.\n", - " with open(f'{data_path}/df_data', 'wb') as f:\n", - " pickle.dump((df_train, target, df_test), f)\n", - " \n", - " return(print('Done!'))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Feature Engineering" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "# feature engineering step\n", - "\n", - "def feature_engineering(data_path):\n", - " \n", - " # install the necessary libraries\n", - " import sys, subprocess;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " \n", - " # import Library\n", - " import os, pickle, gc\n", - " import numpy as np\n", - " import pandas as pd\n", - "\n", - " # loading data\n", - " with open(f'{data_path}/df_data', 'rb') as f:\n", - " df_train, target, df_test = pickle.load(f)\n", - " \n", - " # feature engineering gotten from https://www.kaggle.com/code/ambrosm/amex-lightgbm-quickstart\n", - " def get_features(df, \n", - " features_avg, \n", - " features_min, \n", - " features_max, \n", - " features_last\n", - " ):\n", - " '''\n", - " This function takes a dataframe with all features and returns the aggregated feature grouped by the customer id.\n", - "\n", - " df - dataframe\n", - " '''\n", - " cid = pd.Categorical(df.pop('customer_ID'), ordered=True) # get customer id\n", - " last = (cid != np.roll(cid, -1)) # mask for last statement of every customer\n", - "\n", - " df_avg = (df\n", - " .groupby(cid)\n", - " .mean()[features_avg]\n", - " .rename(columns={f: f\"{f}_avg\" for f in features_avg})\n", - " ) \n", - "\n", - " df_min = (df\n", - " .groupby(cid)\n", - " .min()[features_min]\n", - " .rename(columns={f: f\"{f}_min\" for f in features_min})\n", - " )\n", - " gc.collect()\n", - " print('Computed min')\n", - "\n", - " df_max = (df\n", - " .groupby(cid)\n", - " .max()[features_max]\n", - " .rename(columns={f: f\"{f}_max\" for f in features_max})\n", - " )\n", - " gc.collect()\n", - " print('Computed max')\n", - "\n", - " df = (df.loc[last, features_last]\n", - " .rename(columns={f: f\"{f}_last\" for f in features_last})\n", - " .set_index(np.asarray(cid[last]))\n", - " )\n", - " gc.collect()\n", - " print('Computed last')\n", - "\n", - " df_ = pd.concat([df, df_min, df_max, df_avg], axis=1, )\n", - "\n", - " del df, df_avg, df_min, df_max, cid, last\n", - "\n", - " return df_\n", - " \n", - " features_avg = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', \n", - " 'B_16', 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_28', 'B_29', 'B_30', \n", - " 'B_32', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', \n", - " 'D_45', 'D_46', 'D_47', 'D_48', 'D_50', 'D_51', 'D_53', 'D_54', 'D_55', 'D_58', 'D_59', 'D_60', 'D_61', \n", - " 'D_62', 'D_65', 'D_66', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_75', 'D_76', 'D_77', 'D_78', \n", - " 'D_80', 'D_82', 'D_84', 'D_86', 'D_91', 'D_92', 'D_94', 'D_96', 'D_103', 'D_104', 'D_108', 'D_112', 'D_113', \n", - " 'D_114', 'D_115', 'D_117', 'D_118', 'D_119', 'D_120', 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', \n", - " 'D_128', 'D_129', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', 'D_140', 'D_141', 'D_142', 'D_144', \n", - " 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_2', 'R_3', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_14', 'R_15', 'R_16', \n", - " 'R_17', 'R_20', 'R_21', 'R_22', 'R_24', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_9', 'S_11', 'S_12', 'S_13', \n", - " 'S_15', 'S_16', 'S_18', 'S_22', 'S_23', 'S_25', 'S_26']\n", - " features_min = ['B_2', 'B_4', 'B_5', 'B_9', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', 'B_19', 'B_20', 'B_28', 'B_29', 'B_33', 'B_36', \n", - " 'B_42', 'D_39', 'D_41', 'D_42', 'D_45', 'D_46', 'D_48', 'D_50', 'D_51', 'D_53', 'D_55', 'D_56', 'D_58', 'D_59', \n", - " 'D_60', 'D_62', 'D_70', 'D_71', 'D_74', 'D_75', 'D_78', 'D_83', 'D_102', 'D_112', 'D_113', 'D_115', 'D_118', 'D_119', \n", - " 'D_121', 'D_122', 'D_128', 'D_132', 'D_140', 'D_141', 'D_144', 'D_145', 'P_2', 'P_3', 'R_1', 'R_27', 'S_3', 'S_5', \n", - " 'S_7', 'S_9', 'S_11', 'S_12', 'S_23', 'S_25']\n", - " features_max = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', \n", - " 'B_18', 'B_19', 'B_21', 'B_23', 'B_24', 'B_25', 'B_29', 'B_30', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_42', 'D_39', \n", - " 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', 'D_48', 'D_49', 'D_50', 'D_52', 'D_55', 'D_56', 'D_58', 'D_59', \n", - " 'D_60', 'D_61', 'D_63', 'D_64', 'D_65', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_76', 'D_77', 'D_78', 'D_80', 'D_82', \n", - " 'D_84', 'D_91', 'D_102', 'D_105', 'D_107', 'D_110', 'D_111', 'D_112', 'D_115', 'D_116', 'D_117', 'D_118', 'D_119', \n", - " 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', 'D_128', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', \n", - " 'D_138', 'D_140', 'D_141', 'D_142', 'D_144', 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_3', 'R_5', 'R_6', 'R_7', 'R_8', \n", - " 'R_10', 'R_11', 'R_14', 'R_17', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_7', 'S_8', 'S_11', 'S_12', 'S_13', 'S_15', 'S_16', \n", - " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", - " features_last = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', \n", - " 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_26', 'B_28', 'B_29', 'B_30', 'B_32', 'B_33', \n", - " 'B_36', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', \n", - " 'D_48', 'D_49', 'D_50', 'D_51', 'D_52', 'D_53', 'D_54', 'D_55', 'D_56', 'D_58', 'D_59', 'D_60', 'D_61', 'D_62', 'D_63', \n", - " 'D_64', 'D_65', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_75', 'D_76', 'D_77', 'D_78', 'D_79', 'D_80', 'D_81', 'D_82', \n", - " 'D_83', 'D_86', 'D_91', 'D_96', 'D_105', 'D_106', 'D_112', 'D_114', 'D_119', 'D_120', 'D_121', 'D_122', 'D_124', 'D_125', \n", - " 'D_126', 'D_127', 'D_130', 'D_131', 'D_132', 'D_133', 'D_134', 'D_138', 'D_140', 'D_141', 'D_142', 'D_145', 'P_2', 'P_3', \n", - " 'P_4', 'R_1', 'R_2', 'R_3', 'R_4', 'R_5', 'R_6', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_12', 'R_13', 'R_14', 'R_15', \n", - " 'R_19', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_8', 'S_9', 'S_11', 'S_12', 'S_13', 'S_16', 'S_19', 'S_20', \n", - " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", - " \n", - " # apply feature engineering function\n", - " train = get_features(df_train, features_avg, features_min, features_max, features_last)\n", - " test = get_features(df_test, features_avg, features_min, features_max, features_last)\n", - "\n", - " # save the feature engineered data as a pickle file to be used by the modeling component.\n", - " with open(f'{data_path}/features_df', 'wb') as f:\n", - " pickle.dump((train, test, target), f)\n", - " \n", - " return(print('Done!')) " - ] - }, - { - "cell_type": "markdown", - "metadata": { - "papermill": { - "duration": 0.01421, - "end_time": "2022-04-17T07:17:13.396620", - "exception": false, - "start_time": "2022-04-17T07:17:13.382410", - "status": "completed" - }, - "tags": [] - }, - "source": [ - "## Modelling\n", - " " - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [], - "source": [ - "# modeling step\n", - "\n", - "def modeling(data_path):\n", - " \n", - " # install the necessary libraries\n", - " import sys, subprocess;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','scikit-learn'])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", - " \n", - " # import Library\n", - " import os, pickle, joblib, warnings;\n", - " import pandas as pd\n", - " import numpy as np\n", - " from sklearn.model_selection import StratifiedKFold\n", - " from lightgbm import LGBMClassifier\n", - " warnings.filterwarnings(\"ignore\")\n", - " \n", - " # loading data\n", - " with open(f'{data_path}/features_df', 'rb') as f:\n", - " train, test, target = pickle.load(f)\n", - " \n", - " # define the evaluation metric\n", - " # From https://www.kaggle.com/competitions/amex-default-prediction/discussion/328020\n", - " def amex_metric(y_true: np.array, y_pred: np.array) -> float:\n", - "\n", - " # count of positives and negatives\n", - " n_pos = y_true.sum()\n", - " n_neg = y_true.shape[0] - n_pos\n", - "\n", - " # sorting by descring prediction values\n", - " indices = np.argsort(y_pred)[::-1]\n", - " preds, target = y_pred[indices], y_true[indices]\n", - "\n", - " # filter the top 4% by cumulative row weights\n", - " weight = 20.0 - target * 19.0\n", - " cum_norm_weight = (weight / weight.sum()).cumsum()\n", - " four_pct_filter = cum_norm_weight <= 0.04\n", - "\n", - " # default rate captured at 4%\n", - " d = target[four_pct_filter].sum() / n_pos\n", - "\n", - " # weighted gini coefficient\n", - " lorentz = (target / n_pos).cumsum()\n", - " gini = ((lorentz - cum_norm_weight) * weight).sum()\n", - "\n", - " # max weighted gini coefficient\n", - " gini_max = 10 * n_neg * (1 - 19 / (n_pos + 20 * n_neg))\n", - "\n", - " # normalized weighted gini coefficient\n", - " g = gini / gini_max\n", - "\n", - " return 0.5 * (g + d)\n", - "\n", - " def lgb_amex_metric(y_true, y_pred):\n", - " \"\"\"The competition metric with lightgbm's calling convention\"\"\"\n", - " return ('amex_metric_score',\n", - " amex_metric(y_true, y_pred),\n", - " True)\n", - " \n", - " # Cross-validation\n", - "\n", - " features = [f for f in train.columns if f != 'customer_ID' and f != 'target']\n", - "\n", - " print(f\"{len(features)} features\")\n", - "\n", - " score_list = [] # lgbm score per fold\n", - " y_pred_list = [] # fold predictions list\n", - "\n", - " # init StratifiedKFold\n", - " kf = StratifiedKFold(n_splits=4)\n", - "\n", - " for fold, (idx_tr, idx_va) in enumerate(kf.split(train, target)):\n", - "\n", - " X_tr, X_va, y_tr, y_va, model = None, None, None, None, None\n", - "\n", - " X_tr = train.iloc[idx_tr][features]\n", - " X_va = train.iloc[idx_va][features]\n", - " y_tr = target[idx_tr]\n", - " y_va = target[idx_va]\n", - "\n", - " # init model\n", - " model = LGBMClassifier(n_estimators=30,\n", - " learning_rate=0.1, \n", - " num_leaves=100,\n", - " random_state=2022)\n", - " # fit model\n", - " model.fit(X_tr, y_tr,\n", - " eval_set = [(X_va, y_va)], \n", - " eval_metric=[lgb_amex_metric],\n", - " verbose = 20,\n", - " early_stopping_rounds=30)\n", - "\n", - " X_tr, y_tr = None, None\n", - "\n", - " # fold validation set predictions\n", - " y_va_pred = model.predict_proba(X_va, raw_score=True)\n", - "\n", - " # model score\n", - " score = amex_metric(y_va, y_va_pred)\n", - "\n", - " print(f\"Score = {score}\")\n", - " score_list.append(score)\n", - "\n", - " # test set predictions\n", - " y_pred_list.append(model.predict_proba(test[features], raw_score=True))\n", - "\n", - " print(f\"Fold {fold}\") \n", - "\n", - " # save model\n", - " joblib.dump(model, f'{data_path}/lgb.jl')\n", - " \n", - " return(print('Done!')) " - ] - }, - { - "cell_type": "markdown", - "metadata": { - "papermill": { - "duration": 0.01428, - "end_time": "2022-04-17T07:17:23.959655", - "exception": false, - "start_time": "2022-04-17T07:17:23.945375", - "status": "completed" - }, - "tags": [] - }, - "source": [ - "## Evaluation" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "# evaluation step\n", - "\n", - "def evaluation_result(data_path, \n", - " metrics_path: OutputPath(str)) -> NamedTuple(\"EvaluationOutput\", [(\"mlpipeline_metrics\", \"Metrics\")]):\n", - " \n", - " # import Library\n", - " import sys, subprocess;\n", - " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", - " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", - " import json;\n", - " from collections import namedtuple\n", - " import joblib\n", - " import lightgbm as lgb\n", - " from lightgbm import LGBMRegressor\n", - " \n", - " # load model\n", - " model = joblib.load(f'{data_path}/lgb.jl')\n", - "\n", - " # model evaluation\n", - " binary_logloss = model.booster_.best_score.get('valid_0').get('binary_logloss')\n", - " amex_metric_score = model.booster_.best_score.get('valid_0').get('amex_metric_score')\n", - " \n", - " # create kubeflow metric metadata for UI \n", - " metrics = {\n", - " 'metrics': [\n", - " {'name': 'binary-logloss',\n", - " 'numberValue': binary_logloss,\n", - " 'format': 'RAW'},\n", - " {'name': 'amex-metric-score',\n", - " 'numberValue': amex_metric_score,\n", - " 'format': 'RAW'}\n", - " ]\n", - " }\n", - " \n", - "\n", - " with open(metrics_path, \"w\") as f:\n", - " json.dump(metrics, f)\n", - "\n", - " output_tuple = namedtuple(\"EvaluationOutput\", [\"mlpipeline_metrics\"])\n", - "\n", - " return output_tuple(json.dumps(metrics))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Create pipeline components \n", - "\n", - "using `create_component_from_func`" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - "# create light weight components\n", - "download_op = comp.create_component_from_func(download_data,base_image=\"python:3.7.1\")\n", - "load_op = comp.create_component_from_func(load_data,base_image=\"python:3.7.1\")\n", - "feature_eng_op = comp.create_component_from_func(feature_engineering,base_image=\"python:3.7.1\")\n", - "modeling_op = comp.create_component_from_func(modeling, base_image=\"python:3.7.1\")\n", - "evaluation_op = comp.create_component_from_func(evaluation_result, base_image=\"python:3.7.1\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Kubeflow pipeline creation" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [], - "source": [ - "# define pipeline\n", - "@dsl.pipeline(name=\"american-express-default-prediction-pipeline\", \n", - " description=\"predicting credit default.\")\n", - "\n", - "# Define parameters to be fed into pipeline\n", - "def american_express_default_prediction_pipeline(\n", - " dataset: str,\n", - " data_path: str\n", - " ):\n", - " # Define volume to share data between components.\n", - " vop = dsl.VolumeOp(\n", - " name=\"create_data_volume\",\n", - " resource_name=\"data-volume\", \n", - " size=\"24Gi\", \n", - " modes=dsl.VOLUME_MODE_RWO)\n", - " \n", - " \n", - " # Create download container.\n", - " download_container = download_op(dataset, data_path)\\\n", - " .add_pvolumes({data_path: vop.volume}).add_pod_label(\"kaggle-secret\", \"true\")\n", - " # Create load container.\n", - " load_container = load_op(data_path)\\\n", - " .add_pvolumes({data_path: download_container.pvolume})\n", - " # Create feature engineering container.\n", - " feat_eng_container = feature_eng_op(data_path)\\\n", - " .add_pvolumes({data_path: load_container.pvolume})\n", - " # Create modeling container.\n", - " modeling_container = modeling_op(data_path)\\\n", - " .add_pvolumes({data_path: feat_eng_container.pvolume})\n", - " # Create prediction container.\n", - " evaluation_container = evaluation_op(data_path).add_pvolumes({data_path: modeling_container.pvolume})" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [], - "source": [ - "# create client that would enable communication with the Pipelines API server \n", - "client = kfp.Client()" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [], - "source": [ - "# arguments\n", - "dataset = \"amex-data-integer-dtypes-parquet-format\"\n", - "data_path = \"/mnt\"" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "Experiment details." - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "Run details." - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "pipeline_func = american_express_default_prediction_pipeline\n", - "\n", - "experiment_name = 'american_express_default_prediction_pipeline_lightweight'\n", - "run_name = pipeline_func.__name__ + ' run'\n", - "\n", - "arguments = {\n", - " \"dataset\": dataset,\n", - " \"data_path\": data_path\n", - " }\n", - "\n", - "# Compile pipeline to generate compressed YAML definition of the pipeline.\n", - "kfp.compiler.Compiler().compile(pipeline_func, \n", - " '{}.zip'.format(experiment_name))\n", - "\n", - "# Submit pipeline directly from pipeline function\n", - "run_result = client.create_run_from_pipeline_func(pipeline_func, \n", - " experiment_name=experiment_name, \n", - " run_name=run_name, \n", - " arguments=arguments\n", - " )\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "kubeflow_notebook": { - "autosnapshot": true, - "experiment": { - "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", - "name": "g-research-crypto-forecasting" - }, - "experiment_name": "g-research-crypto-forecasting", - "katib_metadata": { - "algorithm": { - "algorithmName": "grid" - }, - "maxFailedTrialCount": 3, - "maxTrialCount": 12, - "objective": { - "objectiveMetricName": "", - "type": "minimize" - }, - "parallelTrialCount": 3, - "parameters": [] - }, - "katib_run": false, - "pipeline_description": "Forecasting short term returns in 14 popular cryptocurrencies.", - "pipeline_name": "g-research-crypto-forecasting-pipeline", - "snapshot_volumes": true, - "steps_defaults": [ - "label:access-ml-pipeline:true", - "label:kaggle-secret:true", - "label:access-rok:true" - ], - "volume_access_mode": "rwm", - "volumes": [ - { - "annotations": [], - "mount_point": "/home/jovyan", - "name": "test-workspace-qtvmt", - "size": 32, - "size_type": "Gi", - "snapshot": false, - "type": "clone" - } - ] - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - }, - "papermill": { - "default_parameters": {}, - "duration": 32.012084, - "end_time": "2022-04-17T07:17:25.053666", - "environment_variables": {}, - "exception": null, - "input_path": "__notebook__.ipynb", - "output_path": "__notebook__.ipynb", - "parameters": {}, - "start_time": "2022-04-17T07:16:53.041582", - "version": "2.3.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# 🪙 American Express - Default Prediction Competition Vanilla KFP Pipeline\n", + "![](./images/background.jpg)\n", + "\n", + "---\n", + "\n", + "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to predict credit default. This competition is hosted by American Express. \n", + "\n", + "> American Express is a globally integrated payments company. The largest payment card issuer in the world, they provide customers with access to products, insights, and experiences that enrich lives and build business success.\n", + "\n", + "The dataset provided is an industrial scale data set of about 5.5 million rows. It has been pre-processed and converted to a lightweight version by raddar for ease of training and better result. This dataset is available in a [parquet format][1].\n", + "\n", + "[1]: https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Install relevant libraries\n", + "\n", + "\n", + ">Update pip `pip install --user --upgrade pip`\n", + "\n", + ">Install and upgrade kubeflow sdk `pip install kfp --upgrade --user --quiet`\n", + "\n", + "You may need to restart your notebook kernel after installing the kfp sdk" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Requirement already satisfied: pip in /usr/local/lib/python3.6/dist-packages (21.3.1)\n" + ] + } + ], + "source": [ + "!pip install --user --upgrade pip" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install kfp --upgrade --user --quiet" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Name: kfp\n", + "Version: 1.8.11\n", + "Summary: KubeFlow Pipelines SDK\n", + "Home-page: https://github.com/kubeflow/pipelines\n", + "Author: The Kubeflow Authors\n", + "Author-email: \n", + "License: UNKNOWN\n", + "Location: /home/jovyan/.local/lib/python3.6/site-packages\n", + "Requires: absl-py, click, cloudpickle, dataclasses, Deprecated, docstring-parser, fire, google-api-python-client, google-auth, google-cloud-storage, jsonschema, kfp-pipeline-spec, kfp-server-api, kubernetes, protobuf, pydantic, PyYAML, requests-toolbelt, strip-hints, tabulate, typer, typing-extensions, uritemplate\n", + "Required-by: kubeflow-kale\n" + ] + } + ], + "source": [ + "# confirm the kfp sdk\n", + "! pip show kfp" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "imports" + ] + }, + "outputs": [], + "source": [ + "import kfp\n", + "import kfp.components as comp\n", + "import kfp.dsl as dsl\n", + "from kfp.components import OutputPath\n", + "from typing import NamedTuple" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# Kubeflow pipeline component creation\n", + "\n", + "## Download the dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "# load data step\n", + "def download_data(dataset, \n", + " data_path):\n", + " \n", + " # install the necessary libraries\n", + " import os, sys, subprocess, zipfile, pickle;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','kaggle'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','wget'])\n", + " \n", + " # import libraries\n", + " import pandas as pd\n", + " import wget\n", + "\n", + " # setup kaggle environment for data download\n", + " with open('/secret/kaggle-secret/password', 'r') as file:\n", + " kaggle_key = file.read().rstrip()\n", + " with open('/secret/kaggle-secret/username', 'r') as file:\n", + " kaggle_user = file.read().rstrip()\n", + " \n", + " os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", + " \n", + " # create data_path directory\n", + " if not os.path.exists(data_path):\n", + " os.makedirs(data_path)\n", + " \n", + " # download kaggle's Amex-credit-prediction data\n", + " subprocess.run([\"kaggle\",\"datasets\", \"download\", \"-d\", f'raddar/{dataset}'])\n", + " \n", + " # extract Amex-credit-prediction.zip to data_path\n", + " with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", + " zip_ref.extractall(data_path)\n", + "\n", + " # download kaggle's Amex-credit-prediction train_labels.zip\n", + " download_link = \"https://github.com/kubeflow/examples/blob/master/american-express-default-kaggle-competition/data/train_labels.zip?raw=true\"\n", + " \n", + " wget.download(download_link, f'{data_path}/train_labels.zip')\n", + "\n", + " # extract Amex-credit-prediction.zip to data_path\n", + " with zipfile.ZipFile(f'{data_path}/train_labels.zip','r') as zip_ref:\n", + " zip_ref.extractall(data_path)\n", + "\n", + " # delete zipfiles\n", + " subprocess.run(['rm', f'{dataset}.zip'])\n", + " subprocess.run(['rm', f'{data_path}/train_labels.zip'])\n", + " return(print('Done!'))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Load Data" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "# load data step\n", + "def load_data(data_path):\n", + " \n", + " # install the necessary libraries\n", + " import os, sys, subprocess, pickle;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pyarrow'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','fastparquet'])\n", + " \n", + " # import libraries\n", + " import pandas as pd\n", + "\n", + " TRAIN_CSV = (f'{data_path}/train.parquet')\n", + " TEST_CSV = f'{data_path}/test.parquet'\n", + " TARGET_CSV = f'{data_path}/train_labels.csv'\n", + " \n", + " # read parquet TRAIN, TEST and TARGET_CSV\n", + " df_train = pd.read_parquet(TRAIN_CSV)\n", + " df_test = pd.read_parquet(TEST_CSV)\n", + " target = pd.read_csv(TARGET_CSV).target.values\n", + " print(f\"target shape: {target.shape}\")\n", + " \n", + " \n", + " # Save all data as a pickle file to be used by the feature_engineering component.\n", + " with open(f'{data_path}/df_data', 'wb') as f:\n", + " pickle.dump((df_train, target, df_test), f)\n", + " \n", + " return(print('Done!'))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Feature Engineering" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "# feature engineering step\n", + "\n", + "def feature_engineering(data_path):\n", + " \n", + " # install the necessary libraries\n", + " import sys, subprocess;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " \n", + " # import Library\n", + " import os, pickle, gc\n", + " import numpy as np\n", + " import pandas as pd\n", + "\n", + " # loading data\n", + " with open(f'{data_path}/df_data', 'rb') as f:\n", + " df_train, target, df_test = pickle.load(f)\n", + " \n", + " # feature engineering gotten from https://www.kaggle.com/code/ambrosm/amex-lightgbm-quickstart\n", + " def get_features(df, \n", + " features_avg, \n", + " features_min, \n", + " features_max, \n", + " features_last\n", + " ):\n", + " '''\n", + " This function takes a dataframe with all features and returns the aggregated feature grouped by the customer id.\n", + "\n", + " df - dataframe\n", + " '''\n", + " cid = pd.Categorical(df.pop('customer_ID'), ordered=True) # get customer id\n", + " last = (cid != np.roll(cid, -1)) # mask for last statement of every customer\n", + "\n", + " df_avg = (df\n", + " .groupby(cid)\n", + " .mean()[features_avg]\n", + " .rename(columns={f: f\"{f}_avg\" for f in features_avg})\n", + " ) \n", + "\n", + " df_min = (df\n", + " .groupby(cid)\n", + " .min()[features_min]\n", + " .rename(columns={f: f\"{f}_min\" for f in features_min})\n", + " )\n", + " gc.collect()\n", + " print('Computed min')\n", + "\n", + " df_max = (df\n", + " .groupby(cid)\n", + " .max()[features_max]\n", + " .rename(columns={f: f\"{f}_max\" for f in features_max})\n", + " )\n", + " gc.collect()\n", + " print('Computed max')\n", + "\n", + " df = (df.loc[last, features_last]\n", + " .rename(columns={f: f\"{f}_last\" for f in features_last})\n", + " .set_index(np.asarray(cid[last]))\n", + " )\n", + " gc.collect()\n", + " print('Computed last')\n", + "\n", + " df_ = pd.concat([df, df_min, df_max, df_avg], axis=1, )\n", + "\n", + " del df, df_avg, df_min, df_max, cid, last\n", + "\n", + " return df_\n", + " \n", + " features_avg = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', \n", + " 'B_16', 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_28', 'B_29', 'B_30', \n", + " 'B_32', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', \n", + " 'D_45', 'D_46', 'D_47', 'D_48', 'D_50', 'D_51', 'D_53', 'D_54', 'D_55', 'D_58', 'D_59', 'D_60', 'D_61', \n", + " 'D_62', 'D_65', 'D_66', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_75', 'D_76', 'D_77', 'D_78', \n", + " 'D_80', 'D_82', 'D_84', 'D_86', 'D_91', 'D_92', 'D_94', 'D_96', 'D_103', 'D_104', 'D_108', 'D_112', 'D_113', \n", + " 'D_114', 'D_115', 'D_117', 'D_118', 'D_119', 'D_120', 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', \n", + " 'D_128', 'D_129', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', 'D_140', 'D_141', 'D_142', 'D_144', \n", + " 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_2', 'R_3', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_14', 'R_15', 'R_16', \n", + " 'R_17', 'R_20', 'R_21', 'R_22', 'R_24', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_9', 'S_11', 'S_12', 'S_13', \n", + " 'S_15', 'S_16', 'S_18', 'S_22', 'S_23', 'S_25', 'S_26']\n", + " features_min = ['B_2', 'B_4', 'B_5', 'B_9', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', 'B_19', 'B_20', 'B_28', 'B_29', 'B_33', 'B_36', \n", + " 'B_42', 'D_39', 'D_41', 'D_42', 'D_45', 'D_46', 'D_48', 'D_50', 'D_51', 'D_53', 'D_55', 'D_56', 'D_58', 'D_59', \n", + " 'D_60', 'D_62', 'D_70', 'D_71', 'D_74', 'D_75', 'D_78', 'D_83', 'D_102', 'D_112', 'D_113', 'D_115', 'D_118', 'D_119', \n", + " 'D_121', 'D_122', 'D_128', 'D_132', 'D_140', 'D_141', 'D_144', 'D_145', 'P_2', 'P_3', 'R_1', 'R_27', 'S_3', 'S_5', \n", + " 'S_7', 'S_9', 'S_11', 'S_12', 'S_23', 'S_25']\n", + " features_max = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', \n", + " 'B_18', 'B_19', 'B_21', 'B_23', 'B_24', 'B_25', 'B_29', 'B_30', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_42', 'D_39', \n", + " 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', 'D_48', 'D_49', 'D_50', 'D_52', 'D_55', 'D_56', 'D_58', 'D_59', \n", + " 'D_60', 'D_61', 'D_63', 'D_64', 'D_65', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_76', 'D_77', 'D_78', 'D_80', 'D_82', \n", + " 'D_84', 'D_91', 'D_102', 'D_105', 'D_107', 'D_110', 'D_111', 'D_112', 'D_115', 'D_116', 'D_117', 'D_118', 'D_119', \n", + " 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', 'D_128', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', \n", + " 'D_138', 'D_140', 'D_141', 'D_142', 'D_144', 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_3', 'R_5', 'R_6', 'R_7', 'R_8', \n", + " 'R_10', 'R_11', 'R_14', 'R_17', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_7', 'S_8', 'S_11', 'S_12', 'S_13', 'S_15', 'S_16', \n", + " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", + " features_last = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', \n", + " 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_26', 'B_28', 'B_29', 'B_30', 'B_32', 'B_33', \n", + " 'B_36', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', \n", + " 'D_48', 'D_49', 'D_50', 'D_51', 'D_52', 'D_53', 'D_54', 'D_55', 'D_56', 'D_58', 'D_59', 'D_60', 'D_61', 'D_62', 'D_63', \n", + " 'D_64', 'D_65', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_75', 'D_76', 'D_77', 'D_78', 'D_79', 'D_80', 'D_81', 'D_82', \n", + " 'D_83', 'D_86', 'D_91', 'D_96', 'D_105', 'D_106', 'D_112', 'D_114', 'D_119', 'D_120', 'D_121', 'D_122', 'D_124', 'D_125', \n", + " 'D_126', 'D_127', 'D_130', 'D_131', 'D_132', 'D_133', 'D_134', 'D_138', 'D_140', 'D_141', 'D_142', 'D_145', 'P_2', 'P_3', \n", + " 'P_4', 'R_1', 'R_2', 'R_3', 'R_4', 'R_5', 'R_6', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_12', 'R_13', 'R_14', 'R_15', \n", + " 'R_19', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_8', 'S_9', 'S_11', 'S_12', 'S_13', 'S_16', 'S_19', 'S_20', \n", + " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", + " \n", + " # apply feature engineering function\n", + " train = get_features(df_train, features_avg, features_min, features_max, features_last)\n", + " test = get_features(df_test, features_avg, features_min, features_max, features_last)\n", + "\n", + " # save the feature engineered data as a pickle file to be used by the modeling component.\n", + " with open(f'{data_path}/features_df', 'wb') as f:\n", + " pickle.dump((train, test, target), f)\n", + " \n", + " return(print('Done!')) " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.01421, + "end_time": "2022-04-17T07:17:13.396620", + "exception": false, + "start_time": "2022-04-17T07:17:13.382410", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## Modelling\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "# modeling step\n", + "\n", + "def modeling(data_path):\n", + " \n", + " # install the necessary libraries\n", + " import sys, subprocess;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','pandas'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','scikit-learn'])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", + " \n", + " # import Library\n", + " import os, pickle, joblib, warnings;\n", + " import pandas as pd\n", + " import numpy as np\n", + " from sklearn.model_selection import StratifiedKFold\n", + " from lightgbm import LGBMClassifier\n", + " warnings.filterwarnings(\"ignore\")\n", + " \n", + " # loading data\n", + " with open(f'{data_path}/features_df', 'rb') as f:\n", + " train, test, target = pickle.load(f)\n", + " \n", + " # define the evaluation metric\n", + " # From https://www.kaggle.com/competitions/amex-default-prediction/discussion/328020\n", + " def amex_metric(y_true: np.array, y_pred: np.array) -> float:\n", + "\n", + " # count of positives and negatives\n", + " n_pos = y_true.sum()\n", + " n_neg = y_true.shape[0] - n_pos\n", + "\n", + " # sorting by descring prediction values\n", + " indices = np.argsort(y_pred)[::-1]\n", + " preds, target = y_pred[indices], y_true[indices]\n", + "\n", + " # filter the top 4% by cumulative row weights\n", + " weight = 20.0 - target * 19.0\n", + " cum_norm_weight = (weight / weight.sum()).cumsum()\n", + " four_pct_filter = cum_norm_weight <= 0.04\n", + "\n", + " # default rate captured at 4%\n", + " d = target[four_pct_filter].sum() / n_pos\n", + "\n", + " # weighted gini coefficient\n", + " lorentz = (target / n_pos).cumsum()\n", + " gini = ((lorentz - cum_norm_weight) * weight).sum()\n", + "\n", + " # max weighted gini coefficient\n", + " gini_max = 10 * n_neg * (1 - 19 / (n_pos + 20 * n_neg))\n", + "\n", + " # normalized weighted gini coefficient\n", + " g = gini / gini_max\n", + "\n", + " return 0.5 * (g + d)\n", + "\n", + " def lgb_amex_metric(y_true, y_pred):\n", + " \"\"\"The competition metric with lightgbm's calling convention\"\"\"\n", + " return ('amex_metric_score',\n", + " amex_metric(y_true, y_pred),\n", + " True)\n", + " \n", + " # Cross-validation\n", + "\n", + " features = [f for f in train.columns if f != 'customer_ID' and f != 'target']\n", + "\n", + " print(f\"{len(features)} features\")\n", + "\n", + " score_list = [] # lgbm score per fold\n", + " y_pred_list = [] # fold predictions list\n", + "\n", + " # init StratifiedKFold\n", + " kf = StratifiedKFold(n_splits=4)\n", + "\n", + " for fold, (idx_tr, idx_va) in enumerate(kf.split(train, target)):\n", + "\n", + " X_tr, X_va, y_tr, y_va, model = None, None, None, None, None\n", + "\n", + " X_tr = train.iloc[idx_tr][features]\n", + " X_va = train.iloc[idx_va][features]\n", + " y_tr = target[idx_tr]\n", + " y_va = target[idx_va]\n", + "\n", + " # init model\n", + " model = LGBMClassifier(n_estimators=30,\n", + " learning_rate=0.1, \n", + " num_leaves=100,\n", + " random_state=2022)\n", + " # fit model\n", + " model.fit(X_tr, y_tr,\n", + " eval_set = [(X_va, y_va)], \n", + " eval_metric=[lgb_amex_metric],\n", + " verbose = 20,\n", + " early_stopping_rounds=30)\n", + "\n", + " X_tr, y_tr = None, None\n", + "\n", + " # fold validation set predictions\n", + " y_va_pred = model.predict_proba(X_va, raw_score=True)\n", + "\n", + " # model score\n", + " score = amex_metric(y_va, y_va_pred)\n", + "\n", + " print(f\"Score = {score}\")\n", + " score_list.append(score)\n", + "\n", + " # test set predictions\n", + " y_pred_list.append(model.predict_proba(test[features], raw_score=True))\n", + "\n", + " print(f\"Fold {fold}\") \n", + "\n", + " # save model\n", + " joblib.dump(model, f'{data_path}/lgb.jl')\n", + " \n", + " return(print('Done!')) " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.01428, + "end_time": "2022-04-17T07:17:23.959655", + "exception": false, + "start_time": "2022-04-17T07:17:23.945375", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## Evaluation" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "# evaluation step\n", + "\n", + "def evaluation_result(data_path, \n", + " metrics_path: OutputPath(str)) -> NamedTuple(\"EvaluationOutput\", [(\"mlpipeline_metrics\", \"Metrics\")]):\n", + " \n", + " # import Library\n", + " import sys, subprocess;\n", + " subprocess.run([\"python\", \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\"])\n", + " subprocess.run([sys.executable, '-m', 'pip', 'install','lightgbm'])\n", + " import json;\n", + " from collections import namedtuple\n", + " import joblib\n", + " import lightgbm as lgb\n", + " from lightgbm import LGBMRegressor\n", + " \n", + " # load model\n", + " model = joblib.load(f'{data_path}/lgb.jl')\n", + "\n", + " # model evaluation\n", + " binary_logloss = model.booster_.best_score.get('valid_0').get('binary_logloss')\n", + " amex_metric_score = model.booster_.best_score.get('valid_0').get('amex_metric_score')\n", + " \n", + " # create kubeflow metric metadata for UI \n", + " metrics = {\n", + " 'metrics': [\n", + " {'name': 'binary-logloss',\n", + " 'numberValue': binary_logloss,\n", + " 'format': 'RAW'},\n", + " {'name': 'amex-metric-score',\n", + " 'numberValue': amex_metric_score,\n", + " 'format': 'RAW'}\n", + " ]\n", + " }\n", + " \n", + "\n", + " with open(metrics_path, \"w\") as f:\n", + " json.dump(metrics, f)\n", + "\n", + " output_tuple = namedtuple(\"EvaluationOutput\", [\"mlpipeline_metrics\"])\n", + "\n", + " return output_tuple(json.dumps(metrics))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create pipeline components \n", + "\n", + "using `create_component_from_func`" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "# create light weight components\n", + "download_op = comp.create_component_from_func(download_data,base_image=\"python:3.7.1\")\n", + "load_op = comp.create_component_from_func(load_data,base_image=\"python:3.7.1\")\n", + "feature_eng_op = comp.create_component_from_func(feature_engineering,base_image=\"python:3.7.1\")\n", + "modeling_op = comp.create_component_from_func(modeling, base_image=\"python:3.7.1\")\n", + "evaluation_op = comp.create_component_from_func(evaluation_result, base_image=\"python:3.7.1\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Kubeflow pipeline creation" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "# define pipeline\n", + "@dsl.pipeline(name=\"american-express-default-prediction-pipeline\", \n", + " description=\"predicting credit default.\")\n", + "\n", + "# Define parameters to be fed into pipeline\n", + "def american_express_default_prediction_pipeline(\n", + " dataset: str,\n", + " data_path: str\n", + " ):\n", + " # Define volume to share data between components.\n", + " vop = dsl.VolumeOp(\n", + " name=\"create_data_volume\",\n", + " resource_name=\"data-volume\", \n", + " size=\"24Gi\", \n", + " modes=dsl.VOLUME_MODE_RWO)\n", + " \n", + " \n", + " # Create download container.\n", + " download_container = download_op(dataset, data_path)\\\n", + " .add_pvolumes({data_path: vop.volume}).add_pod_label(\"kaggle-secret\", \"true\")\n", + " # Create load container.\n", + " load_container = load_op(data_path)\\\n", + " .add_pvolumes({data_path: download_container.pvolume})\n", + " # Create feature engineering container.\n", + " feat_eng_container = feature_eng_op(data_path)\\\n", + " .add_pvolumes({data_path: load_container.pvolume})\n", + " # Create modeling container.\n", + " modeling_container = modeling_op(data_path)\\\n", + " .add_pvolumes({data_path: feat_eng_container.pvolume})\n", + " # Create prediction container.\n", + " evaluation_container = evaluation_op(data_path).add_pvolumes({data_path: modeling_container.pvolume})" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "# create client that would enable communication with the Pipelines API server \n", + "client = kfp.Client()" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "# arguments\n", + "dataset = \"amex-data-integer-dtypes-parquet-format\"\n", + "data_path = \"/mnt\"" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "Experiment details." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "Run details." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "pipeline_func = american_express_default_prediction_pipeline\n", + "\n", + "experiment_name = 'american_express_default_prediction_pipeline_lightweight'\n", + "run_name = pipeline_func.__name__ + ' run'\n", + "\n", + "arguments = {\n", + " \"dataset\": dataset,\n", + " \"data_path\": data_path\n", + " }\n", + "\n", + "# Compile pipeline to generate compressed YAML definition of the pipeline.\n", + "kfp.compiler.Compiler().compile(pipeline_func, \n", + " '{}.zip'.format(experiment_name))\n", + "\n", + "# Submit pipeline directly from pipeline function\n", + "run_result = client.create_run_from_pipeline_func(pipeline_func, \n", + " experiment_name=experiment_name, \n", + " run_name=run_name, \n", + " arguments=arguments\n", + " )\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "kubeflow_notebook": { + "autosnapshot": true, + "experiment": { + "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", + "name": "g-research-crypto-forecasting" + }, + "experiment_name": "g-research-crypto-forecasting", + "katib_metadata": { + "algorithm": { + "algorithmName": "grid" + }, + "maxFailedTrialCount": 3, + "maxTrialCount": 12, + "objective": { + "objectiveMetricName": "", + "type": "minimize" + }, + "parallelTrialCount": 3, + "parameters": [] + }, + "katib_run": false, + "pipeline_description": "Forecasting short term returns in 14 popular cryptocurrencies.", + "pipeline_name": "g-research-crypto-forecasting-pipeline", + "snapshot_volumes": true, + "steps_defaults": [ + "label:access-ml-pipeline:true", + "label:kaggle-secret:true", + "label:access-rok:true" + ], + "volume_access_mode": "rwm", + "volumes": [ + { + "annotations": [], + "mount_point": "/home/jovyan", + "name": "test-workspace-qtvmt", + "size": 32, + "size_type": "Gi", + "snapshot": false, + "type": "clone" + } + ] + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + }, + "papermill": { + "default_parameters": {}, + "duration": 32.012084, + "end_time": "2022-04-17T07:17:25.053666", + "environment_variables": {}, + "exception": null, + "input_path": "__notebook__.ipynb", + "output_path": "__notebook__.ipynb", + "parameters": {}, + "start_time": "2022-04-17T07:16:53.041582", + "version": "2.3.4" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/american-express-default-kaggle-competition/american-express-default-prediction-orig.ipynb b/american-express-default-kaggle-competition/american-express-default-prediction-orig.ipynb index 9c73415b..31f93f1d 100644 --- a/american-express-default-kaggle-competition/american-express-default-prediction-orig.ipynb +++ b/american-express-default-kaggle-competition/american-express-default-prediction-orig.ipynb @@ -1,872 +1,872 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "# 🪙 American Express - Default Prediction Competition Original Notebook\n", - "![](./images/background.jpg)\n", - "\n", - "---\n", - "\n", - "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to predict credit default. This competition is hosted by American Express. \n", - "\n", - "> American Express is a globally integrated payments company. The largest payment card issuer in the world, they provide customers with access to products, insights, and experiences that enrich lives and build business success.\n", - "\n", - "The dataset provided is an industrial scale data set of about 5.5 million rows. It has been pre-processed and converted to a lightweight version by raddar for ease of training and better result. This dataset is available in a [parquet format][1].\n", - "\n", - "[1]: https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Install necessary packages\n", - "\n", - "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", - "\n", - "NOTE: After installing python packages, restart notebook kernel before proceeding." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "!pip install -r requirements.txt --user --quiet" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Imports\n", - "\n", - "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "tags": [ - "imports" - ] - }, - "outputs": [], - "source": [ - "import numpy as np\n", - "import pandas as pd\n", - "import os, subprocess\n", - "import random, zipfile, joblib\n", - "import scipy.stats\n", - "import warnings\n", - "import gc, wget\n", - "\n", - "from sklearn.model_selection import StratifiedKFold\n", - "from lightgbm import LGBMClassifier, log_evaluation\n", - "\n", - "warnings.filterwarnings(\"ignore\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Project hyper-parameters\n", - "\n", - "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "tags": [ - "pipeline-parameters" - ] - }, - "outputs": [], - "source": [ - "# Hyper-parameters\n", - "N_EST = 30\n", - "LR = 0.1" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "Set random seed for reproducibility" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "def fix_all_seeds(seed):\n", - " np.random.seed(seed)\n", - " random.seed(seed)\n", - " os.environ['PYTHONHASHSEED'] = str(seed)\n", - "\n", - "fix_all_seeds(2022)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Download data\n", - "\n", - "In this section, we download the data from kaggle using the Kaggle API credentials" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "tags": [ - "block:download_data" - ] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "CompletedProcess(args=['kaggle', 'datasets', 'download', '-d', 'raddar/amex-data-integer-dtypes-parquet-format'], returncode=0)" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# setup kaggle environment for data download\n", - "dataset = \"amex-data-integer-dtypes-parquet-format\"\n", - "\n", - "# setup kaggle environment for data download\n", - "with open('/secret/kaggle-secret/password', 'r') as file:\n", - " kaggle_key = file.read().rstrip()\n", - "with open('/secret/kaggle-secret/username', 'r') as file:\n", - " kaggle_user = file.read().rstrip()\n", - "\n", - "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", - "\n", - "# download kaggle's Amex-credit-prediction data\n", - "subprocess.run([\"kaggle\",\"datasets\", \"download\", \"-d\", f'raddar/{dataset}'])" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "tags": [ - "block:" - ] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "CompletedProcess(args=['rm', 'data/train_labels.zip'], returncode=0)" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# path to download to\n", - "data_path = 'data'\n", - "\n", - "# extract Amex-credit-prediction.zip to data_path\n", - "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", - " zip_ref.extractall(data_path)\n", - " \n", - "# download kaggle's Amex-credit-prediction train_labels.zip\n", - "download_link = \"https://github.com/kubeflow/examples/blob/master/american-express-default-kaggle-competition/data/train_labels.zip?raw=true\"\n", - "wget.download(download_link, f'{data_path}/train_labels.zip')\n", - "\n", - "# extract Amex-credit-prediction.zip to data_path\n", - "with zipfile.ZipFile(f'{data_path}/train_labels.zip','r') as zip_ref:\n", - " zip_ref.extractall(data_path)\n", - " \n", - "# delete zipfiles\n", - "subprocess.run(['rm', f'{dataset}.zip'])\n", - "subprocess.run(['rm', f'{data_path}/train_labels.zip'])" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Load the dataset\n", - "\n", - "First, let us load and analyze the data.\n", - "\n", - "The data is in csv format, thus, we use the handy read_csv pandas method." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "tags": [ - "block:load_data", - "prev:download_data" - ] - }, - "outputs": [], - "source": [ - "TRAIN_CSV = (f'{data_path}/train.parquet')\n", - "TEST_CSV = f'{data_path}/test.parquet'\n", - "TARGET_CSV = f'{data_path}/train_labels.csv'" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "target shape: (458913,)\n" - ] - } - ], - "source": [ - "df_train = pd.read_parquet(TRAIN_CSV)\n", - "df_test = pd.read_parquet(TEST_CSV)\n", - "target = pd.read_csv(TARGET_CSV).target.values\n", - "print(f\"target shape: {target.shape}\")" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "(5531451, 190)" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "customer_ID 0\n", - "S_2 0\n", - "P_2 45985\n", - "D_39 0\n", - "B_1 0\n", - " ... \n", - "D_141 101548\n", - "D_142 4587043\n", - "D_143 0\n", - "D_144 40727\n", - "D_145 0\n", - "Length: 190, dtype: int64" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "df_train.isna().sum()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "### Define Helper Functions" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "tags": [ - "functions" - ] - }, - "outputs": [], - "source": [ - "# @yunchonggan's fast metric implementation\n", - "# From https://www.kaggle.com/competitions/amex-default-prediction/discussion/328020\n", - "def amex_metric(y_true: np.array, y_pred: np.array) -> float:\n", - "\n", - " # count of positives and negatives\n", - " n_pos = y_true.sum()\n", - " n_neg = y_true.shape[0] - n_pos\n", - "\n", - " # sorting by descring prediction values\n", - " indices = np.argsort(y_pred)[::-1]\n", - " preds, target = y_pred[indices], y_true[indices]\n", - "\n", - " # filter the top 4% by cumulative row weights\n", - " weight = 20.0 - target * 19.0\n", - " cum_norm_weight = (weight / weight.sum()).cumsum()\n", - " four_pct_filter = cum_norm_weight <= 0.04\n", - "\n", - " # default rate captured at 4%\n", - " d = target[four_pct_filter].sum() / n_pos\n", - "\n", - " # weighted gini coefficient\n", - " lorentz = (target / n_pos).cumsum()\n", - " gini = ((lorentz - cum_norm_weight) * weight).sum()\n", - "\n", - " # max weighted gini coefficient\n", - " gini_max = 10 * n_neg * (1 - 19 / (n_pos + 20 * n_neg))\n", - "\n", - " # normalized weighted gini coefficient\n", - " g = gini / gini_max\n", - "\n", - " return 0.5 * (g + d)\n", - "\n", - "def lgb_amex_metric(y_true, y_pred):\n", - " \"\"\"The competition metric with lightgbm's calling convention\"\"\"\n", - " return ('amex_metric_score',\n", - " amex_metric(y_true, y_pred),\n", - " True)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Feature Engineering" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "features_avg = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', \n", - " 'B_16', 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_28', 'B_29', 'B_30', \n", - " 'B_32', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', \n", - " 'D_45', 'D_46', 'D_47', 'D_48', 'D_50', 'D_51', 'D_53', 'D_54', 'D_55', 'D_58', 'D_59', 'D_60', 'D_61', \n", - " 'D_62', 'D_65', 'D_66', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_75', 'D_76', 'D_77', 'D_78', \n", - " 'D_80', 'D_82', 'D_84', 'D_86', 'D_91', 'D_92', 'D_94', 'D_96', 'D_103', 'D_104', 'D_108', 'D_112', 'D_113', \n", - " 'D_114', 'D_115', 'D_117', 'D_118', 'D_119', 'D_120', 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', \n", - " 'D_128', 'D_129', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', 'D_140', 'D_141', 'D_142', 'D_144', \n", - " 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_2', 'R_3', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_14', 'R_15', 'R_16', \n", - " 'R_17', 'R_20', 'R_21', 'R_22', 'R_24', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_9', 'S_11', 'S_12', 'S_13', \n", - " 'S_15', 'S_16', 'S_18', 'S_22', 'S_23', 'S_25', 'S_26']\n", - "features_min = ['B_2', 'B_4', 'B_5', 'B_9', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', 'B_19', 'B_20', 'B_28', 'B_29', 'B_33', 'B_36', \n", - " 'B_42', 'D_39', 'D_41', 'D_42', 'D_45', 'D_46', 'D_48', 'D_50', 'D_51', 'D_53', 'D_55', 'D_56', 'D_58', 'D_59', \n", - " 'D_60', 'D_62', 'D_70', 'D_71', 'D_74', 'D_75', 'D_78', 'D_83', 'D_102', 'D_112', 'D_113', 'D_115', 'D_118', 'D_119', \n", - " 'D_121', 'D_122', 'D_128', 'D_132', 'D_140', 'D_141', 'D_144', 'D_145', 'P_2', 'P_3', 'R_1', 'R_27', 'S_3', 'S_5', \n", - " 'S_7', 'S_9', 'S_11', 'S_12', 'S_23', 'S_25']\n", - "features_max = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', \n", - " 'B_18', 'B_19', 'B_21', 'B_23', 'B_24', 'B_25', 'B_29', 'B_30', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_42', 'D_39', \n", - " 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', 'D_48', 'D_49', 'D_50', 'D_52', 'D_55', 'D_56', 'D_58', 'D_59', \n", - " 'D_60', 'D_61', 'D_63', 'D_64', 'D_65', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_76', 'D_77', 'D_78', 'D_80', 'D_82', \n", - " 'D_84', 'D_91', 'D_102', 'D_105', 'D_107', 'D_110', 'D_111', 'D_112', 'D_115', 'D_116', 'D_117', 'D_118', 'D_119', \n", - " 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', 'D_128', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', \n", - " 'D_138', 'D_140', 'D_141', 'D_142', 'D_144', 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_3', 'R_5', 'R_6', 'R_7', 'R_8', \n", - " 'R_10', 'R_11', 'R_14', 'R_17', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_7', 'S_8', 'S_11', 'S_12', 'S_13', 'S_15', 'S_16', \n", - " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", - "features_last = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', \n", - " 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_26', 'B_28', 'B_29', 'B_30', 'B_32', 'B_33', \n", - " 'B_36', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', \n", - " 'D_48', 'D_49', 'D_50', 'D_51', 'D_52', 'D_53', 'D_54', 'D_55', 'D_56', 'D_58', 'D_59', 'D_60', 'D_61', 'D_62', 'D_63', \n", - " 'D_64', 'D_65', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_75', 'D_76', 'D_77', 'D_78', 'D_79', 'D_80', 'D_81', 'D_82', \n", - " 'D_83', 'D_86', 'D_91', 'D_96', 'D_105', 'D_106', 'D_112', 'D_114', 'D_119', 'D_120', 'D_121', 'D_122', 'D_124', 'D_125', \n", - " 'D_126', 'D_127', 'D_130', 'D_131', 'D_132', 'D_133', 'D_134', 'D_138', 'D_140', 'D_141', 'D_142', 'D_145', 'P_2', 'P_3', \n", - " 'P_4', 'R_1', 'R_2', 'R_3', 'R_4', 'R_5', 'R_6', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_12', 'R_13', 'R_14', 'R_15', \n", - " 'R_19', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_8', 'S_9', 'S_11', 'S_12', 'S_13', 'S_16', 'S_19', 'S_20', \n", - " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "tags": [ - "block:feature_engineering", - "prev:load_data" - ] - }, - "outputs": [], - "source": [ - "# feature engineering gotten from https://www.kaggle.com/code/ambrosm/amex-lightgbm-quickstart\n", - "def get_features(df, \n", - " features_avg, \n", - " features_min, \n", - " features_max, \n", - " features_last\n", - " ):\n", - " '''\n", - " This function takes a dataframe with all features and returns the aggregated feature grouped by the customer id.\n", - " \n", - " df - dataframe\n", - " '''\n", - " cid = pd.Categorical(df.pop('customer_ID'), ordered=True) # get customer id\n", - " last = (cid != np.roll(cid, -1)) # mask for last statement of every customer\n", - " \n", - " df_avg = (df\n", - " .groupby(cid)\n", - " .mean()[features_avg]\n", - " .rename(columns={f: f\"{f}_avg\" for f in features_avg})\n", - " ) \n", - " \n", - " df_min = (df\n", - " .groupby(cid)\n", - " .min()[features_min]\n", - " .rename(columns={f: f\"{f}_min\" for f in features_min})\n", - " )\n", - " gc.collect()\n", - " print('Computed min')\n", - " \n", - " df_max = (df\n", - " .groupby(cid)\n", - " .max()[features_max]\n", - " .rename(columns={f: f\"{f}_max\" for f in features_max})\n", - " )\n", - " gc.collect()\n", - " print('Computed max')\n", - " \n", - " df = (df.loc[last, features_last]\n", - " .rename(columns={f: f\"{f}_last\" for f in features_last})\n", - " .set_index(np.asarray(cid[last]))\n", - " )\n", - " gc.collect()\n", - " print('Computed last')\n", - " \n", - " df_ = pd.concat([df, df_min, df_max, df_avg], axis=1, )\n", - " \n", - " del df, df_avg, df_min, df_max, cid, last\n", - " \n", - " return df_" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Computed min\n", - "Computed max\n", - "Computed last\n", - "Computed min\n", - "Computed max\n", - "Computed last\n" - ] - } - ], - "source": [ - "# apply feature engineering function\n", - "train = get_features(df_train, features_avg, features_min, features_max, features_last)\n", - "test = get_features(df_test, features_avg, features_min, features_max, features_last)" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "B_1_last False\n", - "B_2_last True\n", - "B_3_last True\n", - "B_4_last False\n", - "B_5_last False\n", - " ... \n", - "S_18_avg False\n", - "S_22_avg True\n", - "S_23_avg True\n", - "S_25_avg True\n", - "S_26_avg False\n", - "Length: 469, dtype: bool" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# check null values\n", - "train.isna().any()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Modelling: StratifiedKFold\n", - "\n", - "We cross-validate with a six-fold StratifiedKFold to handle the imbalanced nature of the target.\n", - "\n", - "Lightgbm handles null values efficiently." - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "tags": [ - "block:modelling", - "prev:feature_engineering" - ] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "469 features\n", - "[20]\tvalid_0's binary_logloss: 0.267976\tvalid_0's amex_metric_score: 0.750976\n", - "Score = 0.7604229987279087\n", - "Fold 0\n", - "[20]\tvalid_0's binary_logloss: 0.267339\tvalid_0's amex_metric_score: 0.753257\n", - "Score = 0.7624180573803372\n", - "Fold 1\n", - "OOF Score: 0.76142\n" - ] - } - ], - "source": [ - "# Cross-validation\n", - "\n", - "features = [f for f in train.columns if f != 'customer_ID' and f != 'target']\n", - "\n", - "print(f\"{len(features)} features\")\n", - "\n", - "score_list = [] # lgbm score per fold\n", - "y_pred_list = [] # fold predictions list\n", - "\n", - "# init StratifiedKFold\n", - "kf = StratifiedKFold(n_splits=4)\n", - "\n", - "for fold, (idx_tr, idx_va) in enumerate(kf.split(train, target)):\n", - " \n", - " X_tr, X_va, y_tr, y_va, model = None, None, None, None, None\n", - "\n", - " X_tr = train.iloc[idx_tr][features]\n", - " X_va = train.iloc[idx_va][features]\n", - " y_tr = target[idx_tr]\n", - " y_va = target[idx_va]\n", - " \n", - " # init model\n", - " model = LGBMClassifier(n_estimators=N_EST,\n", - " learning_rate=LR, \n", - " random_state=2022)\n", - " # fit model\n", - " model.fit(X_tr, y_tr,\n", - " eval_set = [(X_va, y_va)], \n", - " eval_metric=[lgb_amex_metric],\n", - " early_stopping_rounds=30,\n", - " callbacks=[log_evaluation(20)])\n", - " \n", - " X_tr, y_tr = None, None\n", - " \n", - " # fold validation set predictions\n", - " y_va_pred = model.predict_proba(X_va, raw_score=True)\n", - " \n", - " # model score\n", - " score = amex_metric(y_va, y_va_pred)\n", - "\n", - " print(f\"Score = {score}\")\n", - " score_list.append(score)\n", - " \n", - " # test set predictions\n", - " y_pred_list.append(model.predict_proba(test[features], raw_score=True))\n", - " \n", - " print(f\"Fold {fold}\") \n", - "\n", - "# save model\n", - "joblib.dump(model, 'lgb.jl')\n", - "print(f\"OOF Score: {np.mean(score_list):.5f}\")" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAABLMAAAI/CAYAAACMIJv7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABda0lEQVR4nO3dfbTdZXnn//eHI5LQkNBK8EcBDShjKk9RN6BQFFELNtSCpWqnWrEPGaZQfIAOsc5U7LTToA6UisoEKVQKlYqiCAp2HGkpVeQEEkIAq0AspCqVh0gkoITr98f+hh4P52Gfx713eL/Wysre9/f63vf1Ze1luq5e9/1NVSFJkiRJkiT1g+26nYAkSZIkSZLUKYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+8axuJ9Dvdtlll1q0aFG305AkSZIkSdpmrFq16gdVtXCkaxazpmjRokUMDg52Ow1JkiRJkqRtRpLvjHbNbYaSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlveGbWFK3dsJFFy6/udhqSJEmSJOkZav2Kpd1OYVb1TGdWki1JVie5Lcmnk+w4StyeSb6a5PYk65K8c5x5L0py/CTyOSLJoRO9T5IkSZIkSTOnZ4pZwOaqWlJV+wE/Bk4cJe4J4NSqejHwcuCkJC+egXyOACxmSZIkSZIk9ZBeKmYNdT3wwpEuVNV3q+rm5vMjwB3A7p1MmuSPk9zUdH+tTJJm/JSm0+vWJJ9Ksoh2Me3dTbfY4dPxUJIkSZIkSZqanjszK8mzgNcD13QQuwh4CXBjh9OfW1V/0tx7MXAM8AVgObBXVT2eZOeqejjJecCmqvrwCOsuA5YBDMxf2OHSkiRJkiRJmqpe6syam2Q1MAj8K3DBWMFJ5gGfAd5VVT/scI1XJ7kxyVrgSGDfZvxW4JIkb6W9jXFMVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqkza3NVLekkMMn2tAtZl1TVZzu8Zw7wMaBVVfcmOQOY01xeCrwS+BXgfUn2n2DukiRJkiRJmgW91JnVkeacqwuAO6rqrAncurVw9YOmq+v4Zr7tgD2r6qvA6cACYB7wCLDTtCUuSZIkSZKkKeu7YhZwGPA24MjmcPbVSX55vJuq6mHgfOA24FrgpubSAPA3zdbDW4C/bGK/ABznAfCSJEmSJEm9I1XV7Rz6WqvVqsHBwW6nIUmSJEmStM1IsqqqWiNd68fOLEmSJEmSJD1D9dIB8D8lyXOAr4xw6TVV9cAI8R+lvQVxqHOq6sKZyE+SJEmSJEmzr2eLWU3BaskE4k+auWwkSZIkSZLUC9xmKEmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqGz37NsN+sXbDRhYtv7rbaUiSJEnST1m/Ymm3U5CkGTFrnVlJtiRZnWRdkjVJTk0y6vpJDm7iVzfxxw259s4ktzVzvWucdS9Kcvwk8j0iyaETvU+SJEmSJEkzZzY7szZX1RKAJLsClwLzgfePEn8b0KqqJ5LsBqxJ8gVgMfB7wMHAj4FrklxVVd+e5nyPADYB/zzN80qSJEmSJGmSunJmVlXdDywDTk6SUWIeraonmq9zgGo+/wJw45Dr/wC8sZN1k/xxkpuarq6VW9dOckqS25PcmuRTSRYBJwLvbjrDDp/0w0qSJEmSJGnadO0A+Kq6GxgAdh0tJskhSdYBa4ETm+LVbcDhSZ6TZEfgl4E9O1z23Ko6qKr2A+YCxzTjy4GXVNUBzTrrgfOAs6tqSVVdPyyvZUkGkwxueXRjx88sSZIkSZKkqenptxlW1Y1VtS9wEPDeJHOq6g7gTODLwDXAamBLh1O+OsmNSdYCRwL7NuO3ApckeSvwxKh3/0deK6uqVVWtgR0XTOyhJEmSJEmSNGldK2Yl2Zt2Eer+8WKbAtYmYL/m+wVV9bKqeiXwEPAvHaw3B/gYcHxV7Q+cT3v7IsBS4KPAS4GbkviWR0mSJEmSpB7UlWJWkoW0t/GdW1U1SsxeW4tKSZ5P++D39c33XZu/n0f7vKxLO1h2a+HqB0nmAcc3c2wH7FlVXwVOBxYA84BHgJ0m83ySJEmSJEmaGbPZgTQ3yWpge9pb+S4Gzhoj/heB5Ul+AjwJ/H5V/aC59pkkzwF+ApxUVQ+Pt3hVPZzkfNpnbn0PuKm5NAD8TZIFQIC/bGK/AFye5FeBPxh+bpYkSZIkSZJmX0ZpjFKHWq1WDQ4OdjsNSZIkSZKkbUaSVVXVGulaTx8AL0mSJEmSJA3V9YPOkxxF++2EQ91TVcdNcJ6PAocNGz6nqi6cSn6SJEmSJEnqHV0vZlXVtcC10zDPSdOQjiRJkiRJknqY2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX2j628z7HdrN2xk0fKru52GJElSz1q/Ymm3U5AkSduQnujMSrIlyeoka5LcnOTQDu6Zn+S+JOeOE7c+yS6TyOmEJD8/0fskSZIkSZI0c3qimAVsrqolVXUg8F7gzzu4538C/ziDOZ0AWMySJEmSJEnqIb1SzBpqPvDQWAFJXgY8F/jyRCZO8rkkq5KsS7KsGRtIclGS25KsTfLuJMcDLeCSpmNs7iSfRZIkSZIkSdOoV87MmptkNTAH2A04crTAJNsB/xt4K/DaCa7z21X1YFOcuinJZ4BFwO5VtV8z/85V9XCSk4HTqmpwwk8jSZIkSZKkGdErnVlbtxkuBo4GPpkko8T+PvDFqrpvEuuckmQN8HVgT2Af4G5g7yQfSXI08MPxJkmyLMlgksEtj26cRBqSJEmSJEmajF7pzHpKVX2tObB9IXD/CCGvAA5P8vvAPODZSTZV1fKx5k1yBO1OrldU1aNJrgPmVNVDSQ4EjgJOBN4E/PY4Oa4EVgLssNs+NYHHkyRJkiRJ0hT0XDEryWJgAHhgpOtV9ZtDYk8AWuMVshoLgIeaQtZi4OXNHLsAP66qzyT5JvA3TfwjwE6TfhBJkiRJkiRNu14pZm09MwsgwNurass0r3ENcGKSO4Bv0t5qCLA7cGFzFhe036YIcBFwXpLNtLu5Nk9zPpIkSZIkSZqgVLlLbiparVYNDnpGvCRJkiRJ0nRJsqqqWiNd65UD4CVJkiRJkqRx9co2w6dJsj9w8bDhx6vqkFHibwR2GDb8tqpaOxP5SZIkSZIkafb1bDGrKUItmUD8iEUuSZIkSZIkbTvcZihJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6hs9+zbDfrF2w0YWLb+622lIkrRNW79iabdTkCRJUo+wM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOk78NUkeTnJVB3Nfl6Q1iZyOTfLiid4nSZIkSZKkmdMTxSxgc1UtqaoDgfcCfz5O/IeAt81wTscCFrMkSZIkSZJ6SK8Us4aaDzw0VkBVfQV4ZKITJ/l4ksEk65J8YMj4iiS3J7k1yYebzrA3AB9qOsZeMNG1JEmSJEmSNP165QD4uUlWA3OA3YAjZ2id91XVg0kGgK8kOQDYABwHLK6qSrJzVT2c5Ergqqq6fPgkSZYBywAG5i+coVQlSZIkSZI0XK90Zm3dZrgYOBr4ZJLMwDpvSnIzcAuwL+1thBuBx4ALkrwReHS8SapqZVW1qqo1sOOCGUhTkiRJkiRJI+mVYtZTquprwC7AtLY8JdkLOA14TVUdAFwNzKmqJ4CDgcuBY4BrpnNdSZIkSZIkTZ9e2Wb4lCSLgQHggWmeej7wI2BjkucCrweuSzIP2LGqvpjkBuDuJv4RYKdpzkGSJEmSJElT0CvFrK1nZgEEeHtVbRktOMn1wGJgXpL7gN+pqmvHWqCq1iS5BbgTuBe4obm0E/D5JHOatd/TjH8KOD/JKcDxVXXXSPPuv/sCBlcs7eQZJUmSJEmSNEU9UcyqqoEJxh8+gdgjhnw+YZSwg0e47wbaZ2pJkiRJkiSpR/TcmVmSJEmSJEnSaHqiM2skSfYHLh42/HhVHTJK/BXAXsOGTx9v+6EkSZIkSZL6R88Ws6pqLbBkAvHHzVw2kiRJkiRJ6gVuM5QkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6Rs8eAN8v1m7YyKLlV3c7DUmSZtX6FUu7nYIkSZKeoezMkiRJkiRJUt/omWJWkvclWZfk1iSrkxwyStwFSdY0cZcnmTfbuUqSJEmSJKk7eqKYleQVwDHAS6vqAOC1wL2jhL+7qg5s4v4VOHmW0pQkSZIkSVKX9UQxC9gN+EFVPQ5QVT+oqn8bKbCqfgiQJMBcoEabNMmvJLkxyS1J/m+S5ybZLsn6JDsPiftWc+0FSb6eZG2SP02yaTofUpIkSZIkSVPTK8WsLwN7JvmXJB9L8qqxgpNcCHwPWAx8ZIzQfwJeXlUvAT4F/LeqehL4PHBcM9chwHeq6vvAOcA5VbU/cN8Y6y9LMphkcMujGzt/SkmSJEmSJE1JTxSzqmoT8DJgGfDvwGVJThgj/h3AzwN3AG8eY+o9gGuTrAX+ENi3Gb9syH1vab4DvAL4dPP50jHWX1lVrapqDey4YIzlJUmSJEmSNJ16opgFUFVbquq6qno/7XOwfm28eNrdVmPFfQQ4t+m0+i/AnGb8a8ALkywEjgU+O8X0JUmSJEmSNAt6opiV5EVJ9hkytAT4zghxSfLCrZ+BNwB3jjH1AmBD8/ntWwerqoArgLOAO6rqgebS1/mP4thbJv4kkiRJkiRJmknP6nYCjXnAR5pD2Z8Avk17y+FwAf46yfzm8xrgv44x7xnAp5M8BPw/YK8h1y4DbgJOGDL2LuBvkrwPuAYY90Cs/XdfwOCKpeOFSZIkSZIkaRr0RDGrqlYBh3YQ9yRw2ATm/Tztw95HujZIuyA21AbaB8ZXkrcAL+p0LUmSJEmSJM28nihm9ZCXAec2WxgfBn67u+lIkiRJkiRpqJ4tZiW5gp/eFghwelVdO0Ls+4BfHzb86ar6s4msWVXXAwdOKFFJkiRJkiTNmp4tZlXVcROI/TNgQoUrSZIkSZIk9Z+eeJuhJEmSJEmS1AmLWZIkSZIkSeobFrMkSZIkSZLUNyxmSZIkSZIkqW/07AHw/WLtho0sWn51t9OQJPWY9SuWdjsFSZIkaZtkZ5YkSZIkSZL6Rk8Us5JsSbI6yZokNyc5dIzY5zcxq5OsS3LiOHOvT7LLJHI6IcnPT/Q+SZIkSZIkzZxe2Wa4uaqWACQ5Cvhz4FWjxH4XeEVVPZ5kHnBbkiur6t+mOacTgNuA6Z5XkiRJkiRJk9Qrxayh5gMPjXaxqn485OsOTKC7LMnngD2BOcA5VbUyyQBwAdACCvgr4N7m+yVJNtMunm2e4HNIkiRJkiRpmvVKMWtuktW0i0y7AUeOFZxkT+Bq4IXAH06gK+u3q+rBJHOBm5J8BlgE7F5V+zVz71xVDyc5GTitqgZHWH8ZsAxgYP7CDpeWJEmSJEnSVPXEmVk02wyrajFwNPDJJBktuKruraoDaBez3p7kuR2uc0qSNcDXaXdo7QPcDeyd5CNJjgZ+ON4kVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqWY9ZSq+hqwCzBuy1PTkXUbcPh4sUmOAF5Le8vggcAtwJyqegg4ELgOOBH4xCRTlyRJkiRJ0gzruWJWksXAAPDAKNf3aLYJkuRngV8EvtnB1AuAh6rq0WaNlzdz7AJsV1WfAf478NIm/hFgp6k8iyRJkiRJkqZXr52ZBRDg7VW1ZZTYXwD+d5JqYj9cVWs7WOMa4MQkd9Aufn29Gd8duDDJ1sLee5u/LwLOG+8A+P13X8DgiqUdLC9JkiRJkqSpSlV1O4e+1mq1anDwaWfES5IkSZIkaZKSrKqq1kjXem6boSRJkiRJkjSaXtlm+DRJ9gcuHjb8eFUdMkr8jcAOw4bf1uEWREmSJEmSJPWBni1mNUWoJROIH7HIJUmSJEmSpG2H2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvtGzB8D3i7UbNrJo+dXdTkOSnjHWr1ja7RQkSZIkdZGdWZIkSZIkSeobs1LMSrIlyeok65KsSXJqknHXTvK8JJuSnNZ83zPJV5Pc3sz1zpnPXpIkSZIkSb1itrYZbq6qJQBJdgUuBeYD7x/nvrOALw35/gRwalXdnGQnYFWSv6+q22cgZ0mSJEmSJPWYWd9mWFX3A8uAk5NktLgkxwL3AOuG3Pvdqrq5+fwIcAew+xhz/F6Sm5pusM8k2THJgiTf2doZluRnktybZPskByW5teki+1CS26bloSVJkiRJkjQtunJmVlXdDQwAu450Pck84HTgA6PNkWQR8BLgxjGW+mxVHVRVB9IufP1OVW0EVgOvamKOAa6tqp8AFwL/peki2zLG2suSDCYZ3PLoxjGWlyRJkiRJ0nTq1QPgzwDOrqpNI11sil2fAd5VVT8cY579klyfZC3wm8C+zfhlwJubz28BLkuyM7BTVX2tGb90tEmramVVtaqqNbDjgk6fSZIkSZIkSVM0W2dm/ZQke9PufLp/lJBDgOOTfBDYGXgyyWNVdW6S7WkXsi6pqs+Os9RFwLFVtSbJCcARzfiVwP9K8nPAy4D/B+w0+SeSJEmSJEnSbJj1YlaShcB5wLlVVSPFVNXhQ+LPADY1hawAFwB3VNVZHSy3E/DdpgD2m8CGZv5NSW4CzgGuqqotwMNJHklySFXdSLtjS5IkSZIkST1ktopZc5OsBran/UbCi2m/qXCiDgPeBqxt5gP4o6r64ijx/4P2mVr/3vw9tPvqMuDT/Ee3FsDvAOcneRL4B8ADsSRJkiRJknpIRmmOekZKMm/rOV1JlgO7VdU7x7qn1WrV4ODgrOQnSZIkSZL0TJBkVVW1RrrWlTOzetjSJO+l/d/lO8AJ3U1HkiRJkiRJQ3W1mJXkKODMYcP3VNVxE5zno7S3IA51TlVdOJF5quoy2tsPJUmSJEmS1IO6WsyqqmuBa6dhnpOmIR1JkiRJkiT1uO26nYAkSZIkSZLUKYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt/o6tsMtwVrN2xk0fKru52GJHXN+hVLu52CJEmSpGeQnunMSvK+JOuS3JpkdZJDRom7IMmaJu7yJPPGmPOMJKdNIpclSX55ovdJkiRJkiRpZvVEMSvJK4BjgJdW1QHAa4F7Rwl/d1Ud2MT9K3DyDKS0BLCYJUmSJEmS1GN6opgF7Ab8oKoeB6iqH1TVv40UWFU/BEgSYC5QnSyQ5PeS3NR0dX0myY7N+K8nua0Z/8ckzwb+BHhz0yH25ml4PkmSJEmSJE2DXilmfRnYM8m/JPlYkleNFZzkQuB7wGLgIx2u8dmqOqiqDgTuAH6nGf9j4Khm/A1V9eNm7LKqWlJVl42w/rIkg0kGtzy6scPlJUmSJEmSNFU9Ucyqqk3Ay4BlwL8DlyU5YYz4dwA/T7so1Wnn1H5Jrk+yFvhNYN9m/AbgoiS/Bwx0mO/KqmpVVWtgxwUdLi9JkiRJkqSp6oliFkBVbamq66rq/bTPwfq18eKBT40XN8RFwMlVtT/wAWBOM8+JwH8H9gRWJXnO5J5AkiRJkiRJM60nillJXpRknyFDS4DvjBCXJC/c+hl4A3Bnh8vsBHw3yfa0O7O2zvmCqrqxqv6YdlfYnsAjTbwkSZIkSZJ6yLO6nUBjHvCRJDsDTwDfpr3lcLgAf51kfvN5DfBfO1zjfwA30i5Y3ch/FKs+1BTSAnylmfNfgeVJVgN/PtK5WZIkSZIkSZp9qeroZYAaRavVqsHBwW6nIUmSJEmStM1IsqqqWiNd64lthpIkSZIkSVInemWb4dMkuQLYa9jw6VV17Qix7wN+fdjwp6vqz2YqP0mSJEmSJM2+ni1mVdVxE4j9M8DClSRJkiRJ0jbObYaSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL7Rs28z7BdrN2xk0fKru52GJE3Z+hVLu52CJEmSJI1r1jqzkmxJsjrJuiRrkpyaZNz1kzwvyaYkpw0Z+6sk9ye5rYP7L0py/CTyPSLJoRO9T5IkSZIkSTNnNrcZbq6qJVW1L/A64PXA+zu47yzgS8PGLgKOnt70nuYIwGKWJEmSJElSD+nKmVlVdT+wDDg5SUaLS3IscA+wbtj9/wg8ONF1k/xxkpuS3JZk5da1k5yS5PYktyb5VJJFwInAu5tussMnupYkSZIkSZKmX9cOgK+qu4EBYNeRrieZB5wOfGAalz23qg6qqv2AucAxzfhy4CVVdQBwYlWtB84Dzm66ya4fltuyJINJBrc8unEa05MkSZIkSdJYevlthmfQLiZtmsY5X53kxiRrgSOBfZvxW4FLkrwVeGK8SapqZVW1qqo1sOOCaUxPkiRJkiRJY+na2wyT7A1sAe4fJeQQ4PgkHwR2Bp5M8lhVnTvJ9eYAHwNaVXVvkjOAOc3lpcArgV8B3pdk/8msIUmSJEmSpJnVlWJWkoW0t/GdW1U1UkxVHT4k/gxg02QLWY2thasfNFsYjwcub96ouGdVfTXJPwFvAeYBjwDzp7CeJEmSJEmSptlsbjOc2xymvg74v8CXmeR5WEn+Fvga8KIk9yX5nfHuqaqHgfOB24BrgZuaSwPA3zRbD28B/rKJ/QJwnAfAS5IkSZIk9Y6M0hilDrVarRocHOx2GpIkSZIkSduMJKuqqjXStV4+AF6SJEmSJEn6KV07AH6rJEcBZw4bvqeqjpvgPB8FDhs2fE5VXTiV/CRJkiRJktQ7ul7MqqpraZ9hNdV5TpqGdCRJkiRJktTD3GYoSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobXX+bYb9bu2Eji5Zf3e00JGnK1q9Y2u0UJEmSJGlcs9aZlWRLktVJ1iVZk+TUJOOun+R5STYlOa35PifJN5o51iX5wDj3X5ekNYl8j03y4oneJ0mSJEmSpJkzm9sMN1fVkqraF3gd8Hrg/R3cdxbwpSHfHweOrKoDgSXA0UlePt3JAscCFrMkSZIkSZJ6SFfOzKqq+4FlwMlJMlpckmOBe4B1Q+6tqtrUfN2++VOdrJvk40kGh3d0JVmR5PYktyb5cJJDgTcAH2q6yV4w0WeUJEmSJEnS9OvamVlVdXeSAWBX4PvDryeZB5xOu4vrtGHXBoBVwAuBj1bVjR0u+76qerC5/ytJDgA2AMcBi6uqkuxcVQ8nuRK4qqouHyG3ZbSLcQzMX9jh0pIkSZIkSZqqXn6b4RnA2UO6sJ5SVVuqagmwB3Bwkv06nPNNSW4GbgH2pb2NcCPwGHBBkjcCj443SVWtrKpWVbUGdlzQ4dKSJEmSJEmaqq51ZiXZG9gC3D9KyCHA8Uk+COwMPJnksao6d2tA00H1VeBo4LZx1tuLdofXQVX1UJKLgDlV9USSg4HXAMcDJwNHTunhJEmSJEmSNCO6UsxKshA4Dzi3qkY876qqDh8SfwawqarObe79SVPImkt7G+KZHSw7H/gRsDHJc2kfQH9ds51xx6r6YpIbgLub+EeAnSb3hJIkSZIkSZoJs1nMmptkNe0D258ALqb9psKJ2g346+bcq+2Av6uqq8a7qarWJLkFuBO4F7ihubQT8Pkkc4AA72nGPwWcn+QU4PiqumsSuUqSJEmSJGkaZZTGKHWo1WrV4OBgt9OQJEmSJEnaZiRZVVWtka718gHwkiRJkiRJ0k/p2gHwWyU5iqefeXVPVR03wXmuAPYaNnx6VV07lfwkSZIkSZLUO7pezGqKTVMuOE20+CVJkiRJkqT+4zZDSZIkSZIk9Q2LWZIkSZIkSeobFrMkSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLf6PrbDPvd2g0bWbT86m6nIUlPWb9iabdTkCRJkqQZM2udWUm2JFmdZF2SNUlOTTLu+kmel2RTktOa73OSfKOZY12SD8x89pIkSZIkSeoFs9mZtbmqlgAk2RW4FJgPvH+c+84CvjTk++PAkVW1Kcn2wD8l+VJVfX0GcpYkSZIkSVIP6cqZWVV1P7AMODlJRotLcixwD7BuyL1VVZuar9s3f2qMOf44yU1JbkuyMm2Lk3xjSMyiJGubz7+c5M4kq5L8ZZKrpvKskiRJkiRJmj5dOwC+qu4GBoBdR7qeZB5wOvC0bYRJBpKsBu4H/r6qbhxjqXOr6qCq2g+YCxxTVXcCz06yVxPzZuCyJHOA/wO8vqpeBiwcJbdlSQaTDG55dGMnjytJkiRJkqRp0MtvMzwDOHtIF9ZTqmpLs2VxD+DgJPuNMc+rk9zYdF4dCezbjP8d7SIWzd+XAYuBu6vqnmb8b0easKpWVlWrqloDOy6Y4GNJkiRJkiRpsrr2NsMkewNbaHdXjeQQ4PgkHwR2Bp5M8lhVnbs1oKoeTvJV4GjgthHWmAN8DGhV1b1JzgDmNJcvAz6d5LPtqepbSZZMy8NJkiRJkiRpRnSlMyvJQuA82lsARzzvqqoOr6pFVbUI+Avgf1XVuUkWJtm5mWcu8DrgzlGW2lq4+kGzbfH4IfPfRbuY9j9oF7YAvgnsnWRR831r55YkSZIkSZJ6wGx2Zs1tzrnaHngCuJj2mwonajfgr5MM0C7G/V1VjXhIe9O5dT7trq3vATcNC7kM+BCwVxO/OcnvA9ck+dEI8ZIkSZIkSeqijNIY9YyVZF5VbWresvhR4FtVdfZo8a1WqwYHB2cvQUmSJEmSpG1cklVV1RrpWi8fAN8tv9d0kK0DFtB+u6EkSZIkSZJ6QNcOgN8qyVHAmcOG76mq4yY4zxU02wWHOL2qrp3IPE0X1qidWJIkSZIkSeqerhezmmLThApOo8wzoeKXJEmSJEmS+o/bDCVJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfaPrbzPsd2s3bGTR8qu7nYYkPWX9iqXdTkGSJEmSZsysdWYl2ZJkdZJ1SdYkOTXJuOsneV6STUlOGzL2ziS3NXO9a5z7L0py/CTyPSLJoRO9T5IkSZIkSTNnNjuzNlfVEoAkuwKXAvOB949z31nAl7Z+SbIf8HvAwcCPgWuSXFVV357mfI8ANgH/PM3zSpIkSZIkaZK6cmZWVd0PLANOTpLR4pIcC9wDrBsy/AvAjVX1aFU9AfwD8MZO1k3yx0luarq6Vm5dO8kpSW5PcmuSTyVZBJwIvLvpJjt8Ms8pSZIkSZKk6dW1A+Cr6m5gANh1pOtJ5gGnAx8Yduk24PAkz0myI/DLwJ4dLntuVR1UVfsBc4FjmvHlwEuq6gDgxKpaD5wHnF1VS6rq+gk8miRJkiRJkmZIL7/N8AzaxaRNQwer6g7gTODLwDXAamBLh3O+OsmNSdYCRwL7NuO3ApckeSvwxHiTJFmWZDDJ4JZHN3a4tCRJkiRJkqaqa8WsJHvTLkLdP0rIIcAHk6wH3gX8UZKTAarqgqp6WVW9EngI+JcO1psDfAw4vqr2B84H5jSXlwIfBV4K3JRkzLPEqmplVbWqqjWw44LxlpYkSZIkSdI0mc0D4J+SZCHtbXznVlWNFFNVhw+JPwPYVFXnNt93rar7kzyP9nlZL+9g2a2Fqx80WxiPBy5v3qi4Z1V9Nck/AW8B5gGP0D6gXpIkSZIkST1iNotZc5OsBranvZXvYtpvKpyMzyR5DvAT4KSqeni8G6rq4STn0z5z63vATc2lAeBvkiwAAvxlE/sF2sWuXwX+wHOzJEmSJEmSui+jNEapQ61WqwYHB7udhiRJkiRJ0jYjyaqqao10rZcPgJckSZIkSZJ+SlfOzBoqyVG030441D1VddwE5/kocNiw4XOq6sKp5CdJkiRJkqTe0fViVlVdC1w7DfOcNA3pSJIkSZIkqYe5zVCSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1De6/jbDfrd2w0YWLb+622lI6mHrVyztdgqSJEmStM2wM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOkbskiRfS7Iuya1J3jzO3NclaU0ip2OTvHii90mSJEmSJGnm9EQxC9hcVUuq6kDgvcCfjxH7KPBbVbUvcDTwF0l2noGcjgUsZkmSJEmSJPWQXilmDTUfeGi0i1X1L1X1rebzvwH3Aws7mTjJx5MMNl1dHxgyviLJ7U2n14ebzrA3AB9qOsZeMKUnkiRJkiRJ0rTolQPg5yZZDcwBdgOO7OSmJAcDzwbu6nCd91XVg0kGgK8kOQDYABwHLK6qSrJzVT2c5Ergqqq6fIR1lwHLAAbmd1RHkyRJkiRJ0jTolc6srdsMF9PeOvjJJBnrhiS7ARcD76iqJztc501JbgZuAfalvY1wI/AYcEGSN9LexjimqlpZVa2qag3suKDDpSVJkiRJkjRVvVLMekpVfQ3YhTG2DiaZD1xNu9Pq653Mm2Qv4DTgNVV1QHP/nKp6AjgYuBw4Brhmak8gSZIkSZKkmdIr2wyfkmQxMAA8MMr1ZwNXAJ8caQvgGOYDPwI2Jnku8HrguiTzgB2r6otJbgDubuIfAXaa5GNIkiRJkiRpBvRKMWvrmVkAAd5eVVtGiX0T8ErgOUlOaMZOqKrVo8QDUFVrktwC3AncC9zQXNoJ+HySOc3a72nGPwWcn+QU4PiqGvFcrv13X8DgiqXjPJ4kSZIkSZKmQ6qq2zn0tVarVYODg91OQ5IkSZIkaZuRZFVVtUa61nNnZkmSJEmSJEmj6ZVthk+TZH/abysc6vGqOmSU+CuAvYYNn15V185EfpIkSZIkSZp9PVvMqqq1wJIJxB83c9lIkiRJkiSpF7jNUJIkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobPXsAfL9Yu2Eji5Zf3e00JPWw9SuWdjsFSZIkSdpm2JklSZIkSZKkvjFrxawkW5KsTrIuyZokpyYZd/0kz0uyKclpQ8Z2TnJ5kjuT3JHkFWPcf1GS4yeR7xFJDp3ofZIkSZIkSZo5s7nNcHNVLQFIsitwKTAfeP84950FfGnY2DnANVV1fJJnAztOc64ARwCbgH+egbklSZIkSZI0CV3ZZlhV9wPLgJOTZLS4JMcC9wDrhowtAF4JXNDM9eOqeriTdZP8cZKbktyWZOXWtZOckuT2JLcm+VSSRcCJwLubbrLDJ/WgkiRJkiRJmlZdOzOrqu4GBoBdR7qeZB5wOvCBYZf2Av4duDDJLUk+keRnOlz23Ko6qKr2A+YCxzTjy4GXVNUBwIlVtR44Dzi7qpZU1fXDcluWZDDJ4JZHN3a4tCRJkiRJkqaqlw+AP4N2MWnTsPFnAS8FPl5VLwF+RLsY1YlXJ7kxyVrgSGDfZvxW4JIkbwWeGG+SqlpZVa2qag3suKDDpSVJkiRJkjRVs3lm1k9JsjewBbh/lJBDgOOTfBDYGXgyyWPA5cB9VXVjE3c5HRSzkswBPga0qureJGcAc5rLS2lvXfwV4H1J9p/UQ0mSJEmSJGlGdaWYlWQh7W1851ZVjRRTVYcPiT8D2FRV5zbf703yoqr6JvAa4PYOlt1auPpBs4XxeODy5o2Ke1bVV5P8E/AWYB7wCO0D6iVJkiRJktQjZrOYNTfJamB72lv5Lqb9psLJ+APa2wKfDdwNvGO8G6rq4STnA7cB3wNuai4NAH/THCwf4C+b2C/QLnb9KvAHw8/N2mr/3RcwuGLpJB9DkiRJkiRJE5FRGqPUoVarVYODg91OQ5IkSZIkaZuRZFVVtUa61ssHwEuSJEmSJEk/pWsHwG+V5CjgzGHD91TVcROc56PAYcOGz6mqC6eSnyRJkiRJknpH14tZVXUtcO00zHPSNKQjSZIkSZKkHuY2Q0mSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvdP0A+H63dsNGFi2/uttpSJoF61cs7XYKkiRJkvSMZ2eWJEmSJEmS+kZPFLOSbEmyOsltSb6QZOdx4q9J8nCSqzqY+7okrUnkdGySF0/0PkmSJEmSJM2cnihmAZuraklV7Qc8CJw0TvyHgLfNcE7HAhazJEmSJEmSekivFLOG+hqw+1gBVfUV4JGJTpzk40kGk6xL8oEh4yuS3J7k1iQfTnIo8AbgQ03H2AsmupYkSZIkSZKmX08dAJ9kAHgNcMEMLfG+qnqwWecrSQ4ANgDHAYurqpLsXFUPJ7kSuKqqLh8hz2XAMoCB+QtnKFVJkiRJkiQN1yudWXOTrAa+BzwX+PsZWudNSW4GbgH2pb2NcCPwGHBBkjcCj443SVWtrKpWVbUGdlwwQ6lKkiRJkiRpuF4pZm2uqiXA84Ew/plZE5ZkL+A04DVVdQBwNTCnqp4ADgYuB44BrpnutSVJkiRJkjQ9eqWYBUBVPQqcApyaZLq3QM4HfgRsTPJc4PUASeYBC6rqi8C7gQOb+EeAnaY5B0mSJEmSJE3BmAWjJO8Z63pVnTW96UBV3ZLkVuA3gItHyet6YDEwL8l9wO9U1bXjzLsmyS3AncC9wA3NpZ2AzyeZQ7srbOszfwo4P8kpwPFVdddI8+6/+wIGVyyd0DNKkiRJkiRpcsbrfpqVzqSqmjfs+6+ME3/4BOY+YsjnE0YJO3iE+26gfaaWJEmSJEmSesSYxayq+sBsJSJJkiRJkiSNp6NzqZLsAXwEOKwZuh54Z1XdN1OJJdmfp28zfLyqDhkl/gpgr2HDp4+3/VCSJEmSJEn9o9ND1i8ELgV+vfn+1mbsdTORFEBVrQWWTCD+uJnKRZIkSZIkSb2h07cZLqyqC6vqiebPRcDCGcxLkiRJkiRJeppOi1kPJHlrkoHmz1uBB2YyMUmSJEmSJGm4TotZvw28Cfge8F3geOCEGcpJkiRJkiRJGlGnZ2b9CfD2qnoIIMnPAR+mXeSSJEmSJEmSZkWnnVkHbC1kAVTVg8BLZiYlSZIkSZIkaWSddmZtl+Rnh3VmdXrvNm3tho0sWn51t9OQnpHWr1ja7RQkSZIkSbOs086s/w18Lcn/TPI/gX8GPjiRhZJsSbI6yZokNyc5dJz4a5I8nOSqYeMnJ/l2kkqyyzhznJDk3Ink2dy3KMl/nuh9kiRJkiRJmlkdFbOq6pPAG4HvN3/eWFUXT3CtzVW1pKoOBN4L/Pk48R8C3jbC+A3Aa4HvTHD9iVgEWMySJEmSJEnqMR1vFayq24Hbp2nd+cBDYwVU1VeSHDHC+C0ASSa0YJJfAf478GzgAeA3q+r7SV4FnLN1euCVwArgF5KsBv66qs6e0GKSJEmSJEmaEbN57tXcpjg0B9gNOHIW1wb4J+DlVVVJfhf4b8CpwGnASVV1Q5J5wGPAcuC0qjpmpImSLAOWAQzMXzgryUuSJEmSJGl2i1mbq2oJQJJXAJ9Msl9V1SytvwdwWZLdaHdn3dOM3wCcleQS4LNVdd94XV9VtRJYCbDDbvvMVv6SJEmSJEnPeJ0eAD+tquprwC7AbLY1fQQ4t6r2B/4L7Q4xqmoF8LvAXOCGJItnMSdJkiRJkiRNwGx2Zj2lKRgN0D67arYsADY0n98+JJcXVNVaYG2Sg4DFwL3ATrOYmyRJkiRJkjowm51Zc5Osbs7Nugx4e1VtGS04yfXAp4HXJLkvyVHN+ClJ7qO9bfDWJJ/ocP0zgE8nWQX8YMj4u5LcluRW4CfAl4BbgS1J1iR598QeU5IkSZIkSTMls3dk1bap1WrV4OBgt9OQJEmSJEnaZiRZVVWtka515cwsSZIkSZIkaTK6cmbWVkn2By4eNvx4VR0ywXneAbxz2PANVXXSVPKTJEmSJElSb+lqMas5eH3JNMxzIXDhlBOSJEmSJElST3OboSRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb3T1bYbbgrUbNrJo+dXdTkN6Rlq/Ymm3U5AkSZIkzbJZ6cxKsiXJ6iTrkqxJcmqScddO8rwkm5Kc1nzfM8lXk9zezPXOacrvn6djHkmSJEmSJM2s2erM2lxVSwCS7ApcCswH3j/OfWcBXxry/Qng1Kq6OclOwKokf19Vt08luao6dCr3S5IkSZIkaXbM+plZVXU/sAw4OUlGi0tyLHAPsG7Ivd+tqpubz48AdwC7jzHHdUnOTjKY5I4kByX5bJJvJfnTIXGbmr+PaO65PMmdSS4ZK0dJkiRJkiTNrq4cAF9VdwMDwK4jXU8yDzgd+MBocyRZBLwEuHGc5X5cVS3gPODzwEnAfsAJSZ4zQvxLgHcBLwb2Bg4bYe1lTYFscMujG8dZXpIkSZIkSdOlV99meAZwdlVtGuliU+z6DPCuqvrhOHNd2fy9FljXdHc9DtwN7DlC/Deq6r6qehJYDSwaHlBVK6uqVVWtgR0XdPI8kiRJkiRJmgZdeZthkr2BLcD9o4QcAhyf5IPAzsCTSR6rqnOTbE+7kHVJVX22g+Ueb/5+csjnrd9Hev6hMVtGiZEkSZIkSVIXzHqhJslC2lv+zq2qGimmqg4fEn8GsKkpZAW4ALijqs6ajXwlSZIkSZLUO2armDU3yWpge9pvJLyY9psKJ+ow4G3A2mY+gD+qqi9OR5KSJEmSJEnqbRmlOUodarVaNTg42O00JEmSJEmSthlJVjUv9HuaXj0AXpIkSZIkSXqarh5unuQo4Mxhw/dU1XETnOejtLcgDnVOVV04lfwkSZIkSZLUW7pazKqqa4Frp2Gek6YhHUmSJEmSJPU4txlKkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvpGV99muC1Yu2Eji5Zf3e00pJ6wfsXSbqcgSZIkSdrG9URnVpItSVYnWZPk5iSHdhi/OsmV48Rel6Q1iZyOTfLiid4nSZIkSZKkmdMrnVmbq2oJQJKjgD8HXtVJ/Aw6FrgKuH2G15EkSZIkSVKHeqIza5j5wEMzMXGSjycZTLIuyQeGjK9IcnuSW5N8uOkMewPwoab76wUzkY8kSZIkSZImplc6s+YmWQ3MAXYDjhwnfk6SQeAJYEVVfa7Ddd5XVQ8mGQC+kuQAYANwHLC4qirJzlX1cLN98aqqunz4JEmWAcsABuYv7HBpSZIkSZIkTVWvdGZtrqolVbUYOBr4ZJKMEf/8qmoB/xn4iwl0Tr0pyc3ALcC+wIuBjcBjwAVJ3gg8Ot4kVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqWY9ZSq+hqwCzBqy1NVbWj+vhu4DnjJePMm2Qs4DXhNVR0AXA3MqaongIOBy4FjgGum+AiSJEmSJEmaIT1XzEqyGBgAHhjl+s8m2aH5vAtwGJ0d0j4f+BGwMclzgdc3c8wDFlTVF4F3Awc28Y8AO03hUSRJkiRJkjTNeu3MLIAAb6+qLaPE/gLwf5I8SbsYt6Kqxi1mVdWaJLcAdwL3Ajc0l3YCPp9kTrP2e5rxTwHnJzkFOL6q7prEc0mSJEmSJGkapaq6nUNfa7VaNTg42O00JEmSJEmSthlJVjXnpT9Nz20zlCRJkiRJkkbTK9sMnybJ/sDFw4Yfr6pDRom/Athr2PDpVXXtTOQnSZIkSZKk2dezxayqWgssmUD8cTOXjSRJkiRJknqB2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX2jZ99m2C/WbtjIouVXdzsNqSesX7G02ylIkiRJkrZxM9aZlWRLktVJ1iVZk+TUJKOul+TgJn51E3/ckGvvTHJbM9e7ZipnSZIkSZIk9baZ7MzaXFVLAJLsClwKzAfeP0r8bUCrqp5IshuwJskXgMXA7wEHAz8GrklyVVV9ewZzlyRJkiRJUg+alTOzqup+YBlwcpKMEvNoVT3RfJ0DVPP5F4Abh1z/B+CNo62V5LokZycZTHJHkoOSfDbJt5L86ZC4zyVZ1XR7LWvGnt/E7ZJkuyTXJ/mlqf8XkCRJkiRJ0nSYtTOzquruJAPArsD3R4pJcgjwV8Dzgbc1XVq3AX+W5DnAZuCXgcFxlvtxVbWSvBP4PPAy4EHgriRnV9UDwG9X1YNJ5gI3JflMVX0nyZnAx4FvALdX1ZdHyHMZ7eIcA/MXTvQ/hSRJkiRJkiapp95mWFU3VtW+wEHAe5PMqao7gDOBLwPXAKuBLeNMdWXz91pgXVV9t6oeB+4G9myunZJkDfD1ZmyfJodP0N4OeSJw2ih5rqyqVlW1BnZcMLmHlSRJkiRJ0oTNWjEryd60i1D3jxfbFLA2Afs13y+oqpdV1SuBh4B/GWeKx5u/nxzyeev3ZyU5Angt8IqqOhC4hfbWRpLsCOzRxM8b98EkSZIkSZI0a2almJVkIXAecG5V1SgxeyV5VvP5+bQPfl/ffN+1+ft5tM/LunSKKS0AHqqqR5MsBl4+5NqZwCXAHwPnT3EdSZIkSZIkTaOZPDNrbpLVwPbAE8DFwFljxP8isDzJT2h3UP1+Vf2gufaZ5sysnwAnVdXDU8ztGuDEJHcA36S91ZAkr6K9xfGwqtqS5NeSvKOqLpziepIkSZIkSZoGGaVRSh1qtVo1ODjeefSSJEmSJEnqVJJVVdUa6VpPHQAvSZIkSZIkjWUmtxmOKMlRtM+lGuqeqjpugvN8FDhs2PA5bgmUJEmSJEnads16MauqrgWunYZ5TpqGdCRJkiRJktRH3GYoSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobs/42w23N2g0bWbT86m6nIfWE9SuWdjsFSZIkSdI2ric6s5JsSbI6yW1JvpBk5zFilyT5WpJ1SW5N8uZx5r4uSWsSOR2b5MUTvU+SJEmSJEkzpyeKWcDmqlpSVfsBDwInjRH7KPBbVbUvcDTwF2MVv6bgWMBiliRJkiRJUg/plWLWUF8Ddh/tYlX9S1V9q/n8b8D9wMJOJk7y8SSDTVfXB4aMr0hye9Pp9eEkhwJvAD7UdIy9YEpPJEmSJEmSpGnRU2dmJRkAXgNc0GH8wcCzgbs6XOJ9VfVgs85XkhwAbACOAxZXVSXZuaoeTnIlcFVVXT7xJ5EkSZIkSdJM6JXOrLlJVgPfA54L/P14NyTZDbgYeEdVPdnhOm9KcjNwC7Av7W2EG4HHgAuSvJH2Nsbx1l7WdHgNbnl0Y4dLS5IkSZIkaap6pZi1uaqWAM8HwthnZpFkPnA17U6rr3eyQJK9gNOA11TVAc39c6rqCeBg4HLgGOCa8eaqqpVV1aqq1sCOCzpZXpIkSZIkSdOgV4pZAFTVo8ApwKlJRtwCmeTZwBXAJye4BXA+8CNgY5LnAq9v5psHLKiqLwLvBg5s4h8BdprUg0iSJEmSJGlG9FQxC6CqbgFuBX5jlJA3Aa8ETmgOZ1+dZEkH866hvb3wTuBS4Ibm0k7AVUluBf4JeE8z/ingD5Pc4gHwkiRJkiRJvSFV1e0c+lqr1arBwcFupyFJkiRJkrTNSLKqqlojXeu5zixJkiRJkiRpNCOeS9ULkuxP+22FQz1eVYeMEn8FsNew4dOr6tqZyE+SJEmSJEmzr2eLWVW1FlgygfjjZi4bSZIkSZIk9QK3GUqSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+kbPvs2wX6zdsJFFy6/udhrStFq/Ymm3U5AkSZIkaUR2ZkmSJEmSJKlvzFgxK8mWJKuTrEuyJsmpScZdL8nzkmxKclrzfc8kX01yezPXO2cqZ0mSJEmSJPW2mdxmuLmqlgAk2RW4FJgPvH+c+84CvjTk+xPAqVV1c5KdgFVJ/r6qbp+BnCVJkiRJktTDZmWbYVXdDywDTk6S0eKSHAvcA6wbcu93q+rm5vMjwB3A7mPMcV2Ss5MMJrkjyUFJPpvkW0n+dEjc55Ksarq9ljVjz2/idkmyXZLrk/zSFB9fkiRJkiRJ02TWDoCvqruTDAC7At8ffj3JPOB04HXAaSPNkWQR8BLgxnGW+3FVtZotiZ8HXgY8CNyV5OyqegD47ap6MMlc4KYkn6mq7yQ5E/g48A3g9qr68gh5LKNdnGNg/sIOnl6SJEmSJEnToZcOgD8DOLuqNo10sSl2fQZ4V1X9cJy5rmz+Xgusa7q7HgfuBvZsrp2SZA3w9WZsH4Cq+gTt7ZAnMkpRrapWVlWrqloDOy7o9PkkSZIkSZI0RbPWmZVkb2ALcP8oIYcAxyf5ILAz8GSSx6rq3CTb0y5kXVJVn+1gucebv58c8nnr92clOQJ4LfCKqno0yXXAnCbPHYE9mvh5wCMdPaAkSZIkSZJm3KwUs5IsBM4Dzq2qGimmqg4fEn8GsKkpZAW4ALijqs6appQWAA81hazFwMuHXDsTuAT4DnA+cMw0rSlJkiRJkqQpmsli1twkq4Htab+R8GLabyqcqMOAtwFrm/kA/qiqvjiF3K4BTkxyB/BN2lsNSfIq4CDgsKrakuTXkryjqi4cbaL9d1/A4IqlU0hFkiRJkiRJncoojVLqUKvVqsHBwW6nIUmSJEmStM1IsqqqWiNd66UD4CVJkiRJkqQxzdoB8FslOYr2uVRD3VNVx01wno/S3oI41DljbQmUJEmSJElSf5v1YlZVXQtcOw3znDQN6UiSJEmSJKmPuM1QkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6huzfgD8tmbtho0sWn51t9OQRrR+xdJupyBJkiRJ0rSyM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOkbsq5vYrX8eS3LsGPHXJWlNIqdjk7x4ovdJkiRJkiRp5vTKNsPNVbUEIMlRwJ8DrxopsKq+CmyN/Tng28CXZyCnY4GrgNtnYG5JkiRJkiRNQk90Zg0zH3iow9jjgS9V1aOdBCf5eJLBJOuSfGDI+Ioktye5NcmHm86wNwAfarq/XjDhp5AkSZIkSdK065XOrLlJVgNzgN2AIzu87y3AWRNY531V9WCSAeArSQ4ANgDHAYurqpLsXFUPJ7kSuKqqLh8+SZJlwDKAgfkLJ7C8JEmSJEmSpqJXOrM2V9WSqloMHA18MknGuiHJbsD+wLUTWOdNSW4GbgH2BV4MbAQeAy5I8kZg3C6vqlpZVa2qag3suGACy0uSJEmSJGkqeqWY9ZSq+hqwCzBey9ObgCuq6iedzJtkL+A04DVVdQBwNTCnqp4ADgYuB44Brpls7pIkSZIkSZpZPVfMSrIYGAAeGCf0N4C/ncDU84EfARuTPBd4fbPePGBBVX0ReDdwYBP/CLDTBOaXJEmSJEnSDOu1M7MAAry9qraMFpxkEbAn8A+dLlBVa5LcAtwJ3Avc0FzaCfh8kjnN2u9pxj8FnJ/kFOD4qrprpHn3330BgyuWdpqGJEmSJEmSpqAnillVNTDB+PXA7h3GHjHk8wmjhB08wn030D5TS5IkSZIkST2i57YZSpIkSZIkSaPpic6skSTZH7h42PDjVXXIKPFXAHsNGz69qibytkNJkiRJkiT1sJ4tZlXVWmDJBOKPm7lsJEmSJEmS1AvcZihJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1jZ49AL5frN2wkUXLr+52GtrGrV+xtNspSJIkSZLUE+zMkiRJkiRJUt+YtWJWki1JVidZl2RNklOTjLt+kucl2ZTktGHjA0luSXLVOPdfl6Q1iXyPTfLiid4nSZIkSZKkmTObnVmbq2pJVe0LvA54PfD+Du47C/jSCOPvBO6YxvyGOxawmCVJkiRJktRDurLNsKruB5YBJyfJaHFJjgXuAdYNG98DWAp8YiLrJvl4ksGmO+wDQ8ZXJLk9ya1JPpzkUOANwIeabrIXTGQdSZIkSZIkzYyuHQBfVXcnGQB2Bb4//HqSecDptLu4Tht2+S+A/wbsNMFl31dVDzbrfiXJAcAG4DhgcVVVkp2r6uEkVwJXVdXlI+S2jHYxjoH5CyeYgiRJkiRJkiarlw+APwM4u6o2DR1Mcgxwf1WtmsScb0pyM3ALsC/tbYQbgceAC5K8EXh0vEmqamVVtaqqNbDjgkmkIUmSJEmSpMnoWmdWkr2BLcD9o4QcAhyf5IPAzsCTSR4DdgfekOSXgTnA/CR/U1VvHWe9vWh3eB1UVQ8luQiYU1VPJDkYeA1wPHAycOSUH1CSJEmSJEnTrivFrCQLgfOAc6uqRoqpqsOHxJ8BbKqqc5uh9zbjRwCnjVfIaswHfgRsTPJc2gfQX9dsZ9yxqr6Y5Abg7ib+ESa+jVGSJEmSJEkzaDaLWXOTrAa2B54ALqb9psJZUVVrktwC3AncC9zQXNoJ+HySOUCA9zTjnwLOT3IKcHxV3TXSvPvvvoDBFUtnNnlJkiRJkiQBkFEao9ShVqtVg4OD3U5DkiRJkiRpm5FkVVW1RrrWywfAS5IkSZIkST+lawfAb5XkKODMYcP3VNVxE5znCmCvYcOnV9W1U8lPkiRJkiRJvaPrxaym2DTlgtNEi1+SJEmSJEnqP24zlCRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvpG1w+A73drN2xk0fKru52GtnHrVyztdgqSJEmSJPUEO7MkSZIkSZLUN2asmJVkS5LVSdYlWZPk1CTjrpfkeUk2JTltyNj6JGub+QZnKmdJkiRJkiT1tpncZri5qpYAJNkVuBSYD7x/nPvOAr40wvirq+oH05qhJEmSJEmS+sqsbDOsqvuBZcDJSTJaXJJjgXuAdZNdK8l1Sc5OMpjkjiQHJflskm8l+dMhcZ9LsqrpHFvWjD2/idslyXZJrk/yS5PNRZIkSZIkSdNr1g6Ar6q7kwwAuwLfH349yTzgdOB1wGnDbwe+nKSA/1NVK8dZ7sdV1UryTuDzwMuAB4G7kpxdVQ8Av11VDyaZC9yU5DNV9Z0kZwIfB74B3F5VXx4h12W0i3MMzF/Y8X8DSZIkSZIkTU0vvc3wDODsqto0QvPWL1bVhma74t8nubOq/nGMua5s/l4LrKuq7wIkuRvYE3gAOCXJcU3cnsA+wANV9Ykkvw6cCCwZafKmmLYSYIfd9qmJPaYkSZIkSZIma9aKWUn2BrYA948ScghwfJIPAjsDTyZ5rKrOraoN0N6umOQK4GBgrGLW483fTw75vPX7s5IcAbwWeEVVPZrkOmBOk+eOwB5N/DzgkQk8piRJkiRJkmbQrBSzkiwEzgPOraoRO5mq6vAh8WcAm6rq3CQ/A2xXVY80n38J+JMpprQAeKgpZC0GXj7k2pnAJcB3gPOBY6a4liRJkiRJkqbJTBaz5iZZDWwPPAFcTPtNhRP1XOCKZuvhs4BLq+qaKeZ2DXBikjuAbwJfB0jyKuAg4LCq2pLk15K8o6ounOJ6kiRJkiRJmgYZpVFKHWq1WjU4ONjtNCRJkiRJkrYZSVZVVWuka9vNdjKSJEmSJEnSZM362wyTHEX7XKqh7qmq40aKH2OejwKHDRs+xy2BkiRJkiRJ265ZL2ZV1bXAtdMwz0nTkI4kSZIkSZL6iNsMJUmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9Y9bfZritWbthI4uWX93tNLSNW79iabdTkCRJkiSpJ8x6Z1aS9yVZl+TWJKuTHDJK3CVJvpnktiR/lWT7ZvxXh9w7mOQXx1hrUZLbJpnnH03mPkmSJEmSJM2cWS1mJXkFcAzw0qo6AHgtcO8o4ZcAi4H9gbnA7zbjXwEOrKolwG8Dn5ihdC1mSZIkSZIk9ZjZ3ma4G/CDqnocoKp+MFpgVX1x6+ck3wD2aMY3DQn7GaA6WTjJIuDi5h6Ak6vqn5PsBlwGzKf93+O/AkuBuUlWA+uq6jc7WUOSJEmSJEkza7a3GX4Z2DPJvyT5WJJXjXdDs73wbcA1Q8aOS3IncDXt7qxO3A+8rqpeCrwZ+Mtm/D8D1zadXgcCq6tqObC5qpaMVMhKsqzZ4ji45dGNHS4vSZIkSZKkqZrVYlbTVfUyYBnw78BlSU4Y57aPAf9YVdcPmeeKqloMHAv8zw6X3x44P8la4NPAi5vxm4B3JDkD2L+qHungOVZWVauqWgM7LuhweUmSJEmSJE3VrB8AX1Vbquq6qno/cDLwa6PFJnk/sBB4zyhz/SOwd5JdOlj63cD3aXdftYBnD5njlcAG4KIkvzWBx5EkSZIkSdIsmu0D4F+UZJ8hQ0uA74wS+7vAUcBvVNWTQ8ZfmCTN55cCOwAPdLD8AuC7zVxvAwaaOZ4PfL+qzqd9mPxLm/ifbH2DoiRJkiRJknrDbB8APw/4SJKdgSeAb9PecjiS82gXur7W1K4+W1V/QruT67eS/ATYDLy5qjo5BP5jwGeazqtrgB8140cAf9jMtwnY2pm1Erg1yc0eAC9JkiRJktQb0lkdSKNptVo1ODjY7TQkSZIkSZK2GUlWVVVrpGuzfmaWJEmSJEmSNFmzvc3waZJcAew1bPj0qrp2AnPsD1w8bPjxqjpkqvlJkiRJkiSpd3S9mFVVx03DHGtpHyYvSZIkSZKkbZjbDCVJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfaPrbzPsd2s3bGTR8qu7nYb6xPoVS7udgiRJkiRJfa0rnVlJtiRZnWRNkpuTHDpG7PObmNVJ1iU5cTZzlSRJkiRJUu/oVmfW5qpaApDkKODPgVeNEvtd4BVV9XiSecBtSa6sqn+bnVQlSZIkSZLUK3rhzKz5wEOjXayqH1fV483XHRgn5ySbknyo6eL6v0kOTnJdkruTvKGJWZTk+qbj66nOsCTHJflK2nZL8i9J/r9pek5JkiRJkiRNUbc6s+YmWQ3MAXYDjhwrOMmewNXAC4E/HKcr62eA/1dVf5jkCuBPgdcBLwb+GrgSuB94XVU9lmQf4G+BVlVdkeTXgJOAo4H3V9X3RshnGbAMYGD+ws6fWpIkSZIkSVPSC9sMXwF8Msl+VVUjBVfVvcABSX4e+FySy6vq+6PM/WPgmubzWuDxqvpJkrXAomZ8e+DcJEuALcB/GnL/HwC3AV+vqr8dJZ+VwEqAHXbbZ8ScJUmSJEmSNP26vs2wqr4G7AKM2+LUdGTdBhw+RthPhhTFngQeb+59kv8o3r0b+D5wINACnj3k/j2a+56bpOv/fSRJkiRJkvQful6sSbIYGAAeGOX6HknmNp9/FvhF4JtTXHYB8N2mwPW2Zn2SPAv4K+A3gDuA90xxHUmSJEmSJE2jbp+ZBRDg7VW1ZZTYXwD+d5JqYj9cVWunuP7HgM8k+S3aWxJ/1Iz/EXB9Vf1TkjXATUmurqo7prieJEmSJEmSpkFGOaZKHWq1WjU4ONjtNCRJkiRJkrYZSVZVVWuka13fZihJkiRJkiR1qlvbDJ8myf7AxcOGH6+qQ0aJvxHYYdjw26ZhC6IkSZIkSZJ6VM8Us5oi1JIJxI9Y5JIkSZIkSdK2y22GkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+0TNvM+xXazdsZNHyq7udhvrE+hVLu52CJEmSJEl9rSc6s5JsSbI6yW1JvpBk5zFin5/k5iZ+XZITZzFVSZIkSZIkdVFPFLOAzVW1pKr2Ax4EThoj9rvAK6pqCXAIsDzJz89CjpIkSZIkSeqyXilmDfU1YPfRLlbVj6vq8ebrDozzDEk+nmSw6eL6QDN2dJJPD4k5IslVzeffSfIvSb6R5Pwk5075iSRJkiRJkjQteqqYlWQAeA1w5Thxeya5FbgXOLOq/m2M8PdVVQs4AHhVkgOA/wsckuRnmpg3A59qOrz+B/By4DBg8SjrL2sKZINbHt04gSeUJEmSJEnSVPRKMWtuktXA94DnAn8/VnBV3VtVBwAvBN6e5LljhL8pyc3ALcC+wIur6gngGuBXkjwLWAp8HjgY+IeqerCqfgJ8eqQJq2plVbWqqjWw44IJPagkSZIkSZImr1eKWZubM7CeD4Sxz8x6StORdRtw+EjXk+wFnAa8pil+XQ3MaS5/CngTcCQwWFWPTOUBJEmSJEmSNPN6pZgFQFU9CpwCnNp0TD1Nkj2SzG0+/yzwi8A3R5lyPvAjYGPTvfX6Idf+AXgp8Hu0C1sAN9Heivizzfq/NsVHkiRJkiRJ0jTqqWIWQFXdAtwK/MYoIb8A3JhkDe2C1Ierau0oc62hvb3wTuBS4IYh17YAV9EucF3VjG0A/hfwjSZ2PeChWJIkSZIkST0iVdXtHHpKknlVtanpzLoC+KuqumK0+FarVYODg7OXoCRJkiRJ0jYuyarmhX5P03OdWT3gjOYw+tuAe4DPdTUbSZIkSZIkPWXEc6l6QZL9gYuHDT9eVYeMEn8jsMOw4beNtgVxNFV12kTiJUmSJEmSNHt6tpjVFKGWTCB+xCKXJEmSJEmSth1uM5QkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Y2efZthv1i7YSOLll/d7TQ0y9avWNrtFCRJkiRJekaalc6sJFuSrE6yLsmaJKcmGXftJM9LsinJaUPGdk5yeZI7k9yR5BUzm70kSZIkSZJ6xWx1Zm2uqiUASXYFLgXmA+8f576zgC8NGzsHuKaqjk/ybGDHac5VkiRJkiRJPWrWz8yqqvuBZcDJSTJaXJJjgXuAdUPGFgCvBC5o5vpxVT08xhy/l+SmphvsM0l2TLIgyXe2doYl+Zkk9ybZPslBSW5tusg+lOS26XhmSZIkSZIkTY+uHABfVXcDA8CuI11PMg84HfjAsEt7Af8OXJjkliSfSPIzYyz12ao6qKoOBO4AfqeqNgKrgVc1MccA11bVT4ALgf/SdJFtmdTDSZIkSZIkacb06tsMzwDOrqpNw8afBbwU+HhVvQT4EbB8jHn2S3J9krXAbwL7NuOXAW9uPr8FuCzJzsBOVfW1ZvzS0SZNsizJYJLBLY9unMBjSZIkSZIkaSq68jbDJHvT7ny6f5SQQ4Djk3wQ2Bl4MsljwOXAfVV1YxN3OWMXsy4Cjq2qNUlOAI5oxq8E/leSnwNeBvw/YKdO86+qlcBKgB1226c6vU+SJEmSJElTM+vFrCQLgfOAc6tqxEJQVR0+JP4MYFNVndt8vzfJi6rqm8BrgNvHWG4n4LtJtqfdmbWhmX9TkptoHyZ/VVVtAR5O8kiSQ5pi2Vum+qySJEmSJEmaXrNVzJqbZDWwPfAEcDHtNxVOxh8AlzRvMrwbeMcYsf8DuJH2OVs38tPdV5cBn+Y/urUAfgc4P8mTwD8A7iGUJEmSJEnqIRmlOeoZKcm8red0JVkO7FZV7xzrnlarVYODg7OSnyRJkiRJ0jNBklVV1RrpWlfOzOphS5O8l/Z/l+8AJ3Q3HUmSJEmSJA3V1WJWkqOAM4cN31NVx01wno8Chw0bPqeqLpzIPFV1Ge3th5IkSZIkSepBXS1mVdW1wLXTMM9J05COJEmSJEmSetx23U5AkiRJkiRJ6pTFLEmSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvdPVthtuCtRs2smj51d1OQ7Ns/Yql3U5BkiRJkqRnJDuzJEmSJEmS1Dd6ppiVZEuS1UluS/LpJDuOEjcnyTeSrEmyLskHxpn3uiStSeRzbJIXT/Q+SZIkSZIkzZyeKWYBm6tqSVXtB/wYOHGUuMeBI6vqQGAJcHSSl89APscCFrMkSZIkSZJ6SC8Vs4a6HnjhSBeqbVPzdfvmT3UyaZKPJxkc3tGVZEWS25PcmuTDSQ4F3gB8qOkWe8HUHkeSJEmSJEnToecOgE/yLOD1wDVjxAwAq2gXvD5aVTd2OP37qurB5v6vJDkA2AAcByyuqkqyc1U9nORK4KqqunyE9ZcBywAG5i+cyONJkiRJkiRpCnqpM2tuktXAIPCvwAWjBVbVlqpaAuwBHJxkvw7XeFOSm4FbgH1pbyPcCDwGXJDkjcCj401SVSurqlVVrYEdF3S4tCRJkiRJkqaqlzqzNjcFqo41HVRfBY4GbhsrNslewGnAQVX1UJKLgDlV9USSg4HXAMcDJwNHTiJ/SZIkSZIkzbBe6szqSJKFSXZuPs8FXgfc2cGt84EfARuTPJf2VkaSzAMWVNUXgXcDBzbxjwA7TW/2kiRJkiRJmope6szq1G7AXzfnXm0H/F1VXTXeTVW1JskttAtf9wI3NJd2Aj6fZA4Q4D3N+KeA85OcAhxfVXeNNO/+uy9gcMXSKT2QJEmSJEnSWH7yk59w33338dhjj3U7lWk1Z84c9thjD7bffvuO70lVRy8C1CharVYNDg52Ow1JkiRJkrQNu+eee9hpp514znOeQ5JupzMtqooHHniARx55hL322uunriVZVVWtke7ru22GkiRJkiRJzzSPPfbYNlXIAkjCc57znAl3m/XsNsMkzwG+MsKl11TVAyPEXwHsNWz49Kq6dibykyRJkiRJmk3bUiFrq8k8U88Ws5qC1ZIJxB83c9lIkiRJkiQ9sx166KH88z//c7fT6N1iliRJkiRJkka2aPnV0zrf+g5ebtcLhSzwzCxJkiRJkiR1YN68eQBcd911vOpVr+JXf/VX2XvvvVm+fDmXXHIJBx98MPvvvz933XUXACeccAInnngirVaL//Sf/hNXXXXVtORhZ5YkSZIkSZImZM2aNdxxxx383M/9HHvvvTe/+7u/yze+8Q3OOeccPvKRj/AXf/EXAKxfv55vfOMb3HXXXbz61a/m29/+NnPmzJnS2nZmSZIkSZIkaUIOOuggdtttN3bYYQde8IIX8Eu/9EsA7L///qxfv/6puDe96U1st9127LPPPuy9997ceeedU17bzqwpWrth47TvU9Xs6WRPsCRJkiRJ+mk77LDDU5+32267p75vt912PPHEE09dG/62wul4I6OdWZIkSZIkSZoRn/70p3nyySe56667uPvuu3nRi1405TlnrDMryRZgLbA98ATwSeDsqnpylPhFwB3AN5uhr1fVic21lwEXAXOBLwLvrKqaqdwlSZIkSZI0dc973vM4+OCD+eEPf8h555035fOyYGa3GW6uqiUASXYFLgXmA+8f4567tt4zzMeB3wNupF3MOhr40nQmK0mSJEmS1C+6cWzOpk2bADjiiCM44ogjnhq/7rrrnvo8/NprX/tazjvvvGnNY1a2GVbV/cAy4ORMcHNkkt2A+VX19aYb65PAsWPEX5fk7CSDSe5IclCSzyb5VpI/HRL3uSSrkqxLsqwZe34Tt0uS7ZJcn+SXJvPMkiRJkiRJmn6zdgB8Vd2dZADYFfj+KGF7JbkF+CHw36vqemB34L4hMfc1Y2P5cVW1krwT+DzwMuBB4K4kZ1fVA8BvV9WDSeYCNyX5TFV9J8mZtDvBvgHcXlVfHj55U/xaBjAwf2Fn/wEkSZIkSZKeQS666KIZmbeX3mb4XeB5VfVAc0bW55LsO8m5rmz+Xgusq6rvAiS5G9gTeAA4JclxTdyewD7AA1X1iSS/DpwILBlp8qpaCawE2GG3fTy7S5IkSZIkaZbMWjEryd7AFuD+ka5X1ePA483nVUnuAv4TsAHYY0joHs3YWB5v/n5yyOet35+V5AjgtcArqurRJNcBc5o8dxyy3jzgkfGfTpIkSZIkaWZVFRM8vannTeb9frNyZlaShcB5wLmjvYUwycJmG+LWwtc+wN1NV9UPk7y8OW/rt2hvHZyKBcBDTSFrMfDyIdfOBC4B/hg4f4rrSJIkSZIkTdmcOXN44IEHJlX86VVVxQMPPDDhNxzOZGfW3CSrge2BJ4CLgbPGiH8l8CdJfkK7g+rEqnqwufb7wEXAXNpvMZzqmwyvAU5McgfwTeDrAEleBRwEHFZVW5L8WpJ3VNWFo020/+4LGOzCGwQkSZIkSdIzxx577MF9993Hv//7v3c7lWk1Z84c9thjj/EDh8i2VNHrhlarVYODg91OQ5IkSZIkaZuRZFVVtUa6NivbDCVJkiRJkqTpMOtvM0xyFO1zqYa6p6qOGyl+jHk+Chw2bPicsbYESpIkSZIkqb/NejGrqq4Frp2GeU6ahnQkSZIkSZLURzwza4qSPEL7EHmpE7sAP+h2Euob/l40Ef5eNBH+XtQpfyuaCH8vmgh/LxrP86tq4UgXZr0zaxv0zdEOJJOGSzLo70Wd8veiifD3oonw96JO+VvRRPh70UT4e9FUeAC8JEmSJEmS+obFLEmSJEmSJPUNi1lTt7LbCaiv+HvRRPh70UT4e9FE+HtRp/ytaCL8vWgi/L1o0jwAXpIkSZIkSX3DzixJkiRJkiT1DYtZU5Dk6CTfTPLtJMu7nY96S5K/SnJ/ktuGjP1ckr9P8q3m75/tZo7qDUn2TPLVJLcnWZfknc24vxc9TZI5Sb6RZE3ze/lAM75Xkhubf5MuS/Lsbueq3pFkIMktSa5qvvt70YiSrE+yNsnqJIPNmP8eaURJdk5yeZI7k9yR5BX+XjSSJC9q/ndl658fJnmXvxdNlsWsSUoyAHwUeD3wYuA3kry4u1mpx1wEHD1sbDnwlaraB/hK8116Aji1ql4MvBw4qfnfE38vGsnjwJFVdSCwBDg6ycuBM4Gzq+qFwEPA73QvRfWgdwJ3DPnu70VjeXVVLamqVvPdf480mnOAa6pqMXAg7f+d8feip6mqbzb/u7IEeBnwKHAF/l40SRazJu9g4NtVdXdV/Rj4FPCrXc5JPaSq/hF4cNjwrwJ/3Xz+a+DY2cxJvamqvltVNzefH6H9fwjujr8XjaDaNjVft2/+FHAkcHkz7u9FT0myB7AU+ETzPfh70cT475GeJskC4JXABQBV9eOqehh/Lxrfa4C7quo7+HvRJFnMmrzdgXuHfL+vGZPG8tyq+m7z+XvAc7uZjHpPkkXAS4Ab8feiUTRbxlYD9wN/D9wFPFxVTzQh/pukof4C+G/Ak8335+DvRaMr4MtJViVZ1oz575FGshfw78CFzTbmTyT5Gfy9aHxvAf62+ezvRZNiMUvqkmq/StTXieopSeYBnwHeVVU/HHrN34uGqqotTZv+HrQ7hRd3NyP1qiTHAPdX1apu56K+8YtV9VLaR2mclOSVQy/675GGeBbwUuDjVfUS4EcM2yLm70XDNWc0vgH49PBr/l40ERazJm8DsOeQ73s0Y9JYvp9kN4Dm7/u7nI96RJLtaReyLqmqzzbD/l40pmY7x1eBVwA7J3lWc8l/k7TVYcAbkqynfSTCkbTPuPH3ohFV1Ybm7/tpn2dzMP57pJHdB9xXVTc23y+nXdzy96KxvB64uaq+33z396JJsZg1eTcB+zRvA3o27VbJK7uck3rflcDbm89vBz7fxVzUI5rzay4A7qiqs4Zc8veip0myMMnOzee5wOton7P2VeD4JszfiwCoqvdW1R5VtYj2/63y/6rqN/H3ohEk+ZkkO239DPwScBv+e6QRVNX3gHuTvKgZeg1wO/5eNLbf4D+2GIK/F01S2p18mowkv0z7HIoB4K+q6s+6m5F6SZK/BY4AdgG+D7wf+Bzwd8DzgO8Ab6qq4YfE6xkmyS8C1wNr+Y8zbf6I9rlZ/l70U5IcQPuA1AHa/0+pv6uqP0myN+3Om58DbgHeWlWPdy9T9ZokRwCnVdUx/l40kuZ3cUXz9VnApVX1Z0meg/8eaQRJltB+ucSzgbuBd9D824S/Fw3TFMn/Fdi7qjY2Y/7viybFYpYkSZIkSZL6htsMJUmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqG/8/Bkfq0YINPYMAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "# feature importance for top 30 features\n", - "fea_imp = pd.DataFrame({'imp':model.feature_importances_, 'col': features})\n", - "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", - "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Evaluation" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "metadata": { - "tags": [ - "block:evaluation_result", - "prev:modelling" - ] - }, - "outputs": [], - "source": [ - "model = joblib.load('lgb.jl')" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "binary_logloss = model.booster_.best_score.get('valid_0').get('binary_logloss')" - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "amex_metric_score = model.booster_.best_score.get('valid_0').get('amex_metric_score')" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.24605493989573005\n" - ] - } - ], - "source": [ - "print(binary_logloss)" - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "metadata": { - "tags": [ - "pipeline-metrics" - ] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.7625788091718922\n" - ] - } - ], - "source": [ - "print(amex_metric_score)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Submission" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "sub = pd.DataFrame({'customer_ID': test.index,\n", - " 'prediction': np.mean(y_pred_list, axis=0)})\n", - "sub.to_csv('submission.csv', index=False)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [ - "skip" - ] - }, - "outputs": [], - "source": [ - "sub" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "kubeflow_notebook": { - "autosnapshot": true, - "experiment": { - "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", - "name": "g-research-crypto-forecasting" - }, - "experiment_name": "g-research-crypto-forecasting", - "katib_metadata": { - "algorithm": { - "algorithmName": "grid" - }, - "maxFailedTrialCount": 3, - "maxTrialCount": 12, - "objective": { - "objectiveMetricName": "", - "type": "minimize" - }, - "parallelTrialCount": 3, - "parameters": [] - }, - "katib_run": false, - "pipeline_description": "forecasting short term returns in 14 popular cryptocurrencies.", - "pipeline_name": "g-research-crypto-forecasting-pipeline", - "snapshot_volumes": true, - "steps_defaults": [ - "label:access-ml-pipeline:true", - "label:kaggle-secret:true", - "label:access-rok:true" - ], - "volume_access_mode": "rwm", - "volumes": [ - { - "annotations": [], - "mount_point": "/home/jovyan", - "name": "test-workspace-qtvmt", - "size": 32, - "size_type": "Gi", - "snapshot": false, - "type": "clone" - } - ] - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "# 🪙 American Express - Default Prediction Competition Original Notebook\n", + "![](./images/background.jpg)\n", + "\n", + "---\n", + "\n", + "In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to predict credit default. This competition is hosted by American Express. \n", + "\n", + "> American Express is a globally integrated payments company. The largest payment card issuer in the world, they provide customers with access to products, insights, and experiences that enrich lives and build business success.\n", + "\n", + "The dataset provided is an industrial scale data set of about 5.5 million rows. It has been pre-processed and converted to a lightweight version by raddar for ease of training and better result. This dataset is available in a [parquet format][1].\n", + "\n", + "[1]: https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Install necessary packages\n", + "\n", + "We can install the necessary package by either running pip install --user or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n", + "\n", + "NOTE: After installing python packages, restart notebook kernel before proceeding." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "!pip install -r requirements.txt --user --quiet" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Imports\n", + "\n", + "In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "imports" + ] + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import os, subprocess\n", + "import random, zipfile, joblib\n", + "import scipy.stats\n", + "import warnings\n", + "import gc, wget\n", + "\n", + "from sklearn.model_selection import StratifiedKFold\n", + "from lightgbm import LGBMClassifier, log_evaluation\n", + "\n", + "warnings.filterwarnings(\"ignore\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Project hyper-parameters\n", + "\n", + "In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "pipeline-parameters" + ] + }, + "outputs": [], + "source": [ + "# Hyper-parameters\n", + "N_EST = 30\n", + "LR = 0.1" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "Set random seed for reproducibility" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "def fix_all_seeds(seed):\n", + " np.random.seed(seed)\n", + " random.seed(seed)\n", + " os.environ['PYTHONHASHSEED'] = str(seed)\n", + "\n", + "fix_all_seeds(2022)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Download data\n", + "\n", + "In this section, we download the data from kaggle using the Kaggle API credentials" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "tags": [ + "block:download_data" + ] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "CompletedProcess(args=['kaggle', 'datasets', 'download', '-d', 'raddar/amex-data-integer-dtypes-parquet-format'], returncode=0)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# setup kaggle environment for data download\n", + "dataset = \"amex-data-integer-dtypes-parquet-format\"\n", + "\n", + "# setup kaggle environment for data download\n", + "with open('/secret/kaggle-secret/password', 'r') as file:\n", + " kaggle_key = file.read().rstrip()\n", + "with open('/secret/kaggle-secret/username', 'r') as file:\n", + " kaggle_user = file.read().rstrip()\n", + "\n", + "os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n", + "\n", + "# download kaggle's Amex-credit-prediction data\n", + "subprocess.run([\"kaggle\",\"datasets\", \"download\", \"-d\", f'raddar/{dataset}'])" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "tags": [ + "block:" + ] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "CompletedProcess(args=['rm', 'data/train_labels.zip'], returncode=0)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# path to download to\n", + "data_path = 'data'\n", + "\n", + "# extract Amex-credit-prediction.zip to data_path\n", + "with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n", + " zip_ref.extractall(data_path)\n", + " \n", + "# download kaggle's Amex-credit-prediction train_labels.zip\n", + "download_link = \"https://github.com/kubeflow/examples/blob/master/american-express-default-kaggle-competition/data/train_labels.zip?raw=true\"\n", + "wget.download(download_link, f'{data_path}/train_labels.zip')\n", + "\n", + "# extract Amex-credit-prediction.zip to data_path\n", + "with zipfile.ZipFile(f'{data_path}/train_labels.zip','r') as zip_ref:\n", + " zip_ref.extractall(data_path)\n", + " \n", + "# delete zipfiles\n", + "subprocess.run(['rm', f'{dataset}.zip'])\n", + "subprocess.run(['rm', f'{data_path}/train_labels.zip'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Load the dataset\n", + "\n", + "First, let us load and analyze the data.\n", + "\n", + "The data is in csv format, thus, we use the handy read_csv pandas method." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "tags": [ + "block:load_data", + "prev:download_data" + ] + }, + "outputs": [], + "source": [ + "TRAIN_CSV = (f'{data_path}/train.parquet')\n", + "TEST_CSV = f'{data_path}/test.parquet'\n", + "TARGET_CSV = f'{data_path}/train_labels.csv'" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "target shape: (458913,)\n" + ] + } + ], + "source": [ + "df_train = pd.read_parquet(TRAIN_CSV)\n", + "df_test = pd.read_parquet(TEST_CSV)\n", + "target = pd.read_csv(TARGET_CSV).target.values\n", + "print(f\"target shape: {target.shape}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(5531451, 190)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "customer_ID 0\n", + "S_2 0\n", + "P_2 45985\n", + "D_39 0\n", + "B_1 0\n", + " ... \n", + "D_141 101548\n", + "D_142 4587043\n", + "D_143 0\n", + "D_144 40727\n", + "D_145 0\n", + "Length: 190, dtype: int64" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_train.isna().sum()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "### Define Helper Functions" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "tags": [ + "functions" + ] + }, + "outputs": [], + "source": [ + "# @yunchonggan's fast metric implementation\n", + "# From https://www.kaggle.com/competitions/amex-default-prediction/discussion/328020\n", + "def amex_metric(y_true: np.array, y_pred: np.array) -> float:\n", + "\n", + " # count of positives and negatives\n", + " n_pos = y_true.sum()\n", + " n_neg = y_true.shape[0] - n_pos\n", + "\n", + " # sorting by descring prediction values\n", + " indices = np.argsort(y_pred)[::-1]\n", + " preds, target = y_pred[indices], y_true[indices]\n", + "\n", + " # filter the top 4% by cumulative row weights\n", + " weight = 20.0 - target * 19.0\n", + " cum_norm_weight = (weight / weight.sum()).cumsum()\n", + " four_pct_filter = cum_norm_weight <= 0.04\n", + "\n", + " # default rate captured at 4%\n", + " d = target[four_pct_filter].sum() / n_pos\n", + "\n", + " # weighted gini coefficient\n", + " lorentz = (target / n_pos).cumsum()\n", + " gini = ((lorentz - cum_norm_weight) * weight).sum()\n", + "\n", + " # max weighted gini coefficient\n", + " gini_max = 10 * n_neg * (1 - 19 / (n_pos + 20 * n_neg))\n", + "\n", + " # normalized weighted gini coefficient\n", + " g = gini / gini_max\n", + "\n", + " return 0.5 * (g + d)\n", + "\n", + "def lgb_amex_metric(y_true, y_pred):\n", + " \"\"\"The competition metric with lightgbm's calling convention\"\"\"\n", + " return ('amex_metric_score',\n", + " amex_metric(y_true, y_pred),\n", + " True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Feature Engineering" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "features_avg = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', \n", + " 'B_16', 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_28', 'B_29', 'B_30', \n", + " 'B_32', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', \n", + " 'D_45', 'D_46', 'D_47', 'D_48', 'D_50', 'D_51', 'D_53', 'D_54', 'D_55', 'D_58', 'D_59', 'D_60', 'D_61', \n", + " 'D_62', 'D_65', 'D_66', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_75', 'D_76', 'D_77', 'D_78', \n", + " 'D_80', 'D_82', 'D_84', 'D_86', 'D_91', 'D_92', 'D_94', 'D_96', 'D_103', 'D_104', 'D_108', 'D_112', 'D_113', \n", + " 'D_114', 'D_115', 'D_117', 'D_118', 'D_119', 'D_120', 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', \n", + " 'D_128', 'D_129', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', 'D_140', 'D_141', 'D_142', 'D_144', \n", + " 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_2', 'R_3', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_14', 'R_15', 'R_16', \n", + " 'R_17', 'R_20', 'R_21', 'R_22', 'R_24', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_9', 'S_11', 'S_12', 'S_13', \n", + " 'S_15', 'S_16', 'S_18', 'S_22', 'S_23', 'S_25', 'S_26']\n", + "features_min = ['B_2', 'B_4', 'B_5', 'B_9', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', 'B_19', 'B_20', 'B_28', 'B_29', 'B_33', 'B_36', \n", + " 'B_42', 'D_39', 'D_41', 'D_42', 'D_45', 'D_46', 'D_48', 'D_50', 'D_51', 'D_53', 'D_55', 'D_56', 'D_58', 'D_59', \n", + " 'D_60', 'D_62', 'D_70', 'D_71', 'D_74', 'D_75', 'D_78', 'D_83', 'D_102', 'D_112', 'D_113', 'D_115', 'D_118', 'D_119', \n", + " 'D_121', 'D_122', 'D_128', 'D_132', 'D_140', 'D_141', 'D_144', 'D_145', 'P_2', 'P_3', 'R_1', 'R_27', 'S_3', 'S_5', \n", + " 'S_7', 'S_9', 'S_11', 'S_12', 'S_23', 'S_25']\n", + "features_max = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', \n", + " 'B_18', 'B_19', 'B_21', 'B_23', 'B_24', 'B_25', 'B_29', 'B_30', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_42', 'D_39', \n", + " 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', 'D_48', 'D_49', 'D_50', 'D_52', 'D_55', 'D_56', 'D_58', 'D_59', \n", + " 'D_60', 'D_61', 'D_63', 'D_64', 'D_65', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_76', 'D_77', 'D_78', 'D_80', 'D_82', \n", + " 'D_84', 'D_91', 'D_102', 'D_105', 'D_107', 'D_110', 'D_111', 'D_112', 'D_115', 'D_116', 'D_117', 'D_118', 'D_119', \n", + " 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', 'D_128', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', \n", + " 'D_138', 'D_140', 'D_141', 'D_142', 'D_144', 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_3', 'R_5', 'R_6', 'R_7', 'R_8', \n", + " 'R_10', 'R_11', 'R_14', 'R_17', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_7', 'S_8', 'S_11', 'S_12', 'S_13', 'S_15', 'S_16', \n", + " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n", + "features_last = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', \n", + " 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_26', 'B_28', 'B_29', 'B_30', 'B_32', 'B_33', \n", + " 'B_36', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', \n", + " 'D_48', 'D_49', 'D_50', 'D_51', 'D_52', 'D_53', 'D_54', 'D_55', 'D_56', 'D_58', 'D_59', 'D_60', 'D_61', 'D_62', 'D_63', \n", + " 'D_64', 'D_65', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_75', 'D_76', 'D_77', 'D_78', 'D_79', 'D_80', 'D_81', 'D_82', \n", + " 'D_83', 'D_86', 'D_91', 'D_96', 'D_105', 'D_106', 'D_112', 'D_114', 'D_119', 'D_120', 'D_121', 'D_122', 'D_124', 'D_125', \n", + " 'D_126', 'D_127', 'D_130', 'D_131', 'D_132', 'D_133', 'D_134', 'D_138', 'D_140', 'D_141', 'D_142', 'D_145', 'P_2', 'P_3', \n", + " 'P_4', 'R_1', 'R_2', 'R_3', 'R_4', 'R_5', 'R_6', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_12', 'R_13', 'R_14', 'R_15', \n", + " 'R_19', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_8', 'S_9', 'S_11', 'S_12', 'S_13', 'S_16', 'S_19', 'S_20', \n", + " 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "tags": [ + "block:feature_engineering", + "prev:load_data" + ] + }, + "outputs": [], + "source": [ + "# feature engineering gotten from https://www.kaggle.com/code/ambrosm/amex-lightgbm-quickstart\n", + "def get_features(df, \n", + " features_avg, \n", + " features_min, \n", + " features_max, \n", + " features_last\n", + " ):\n", + " '''\n", + " This function takes a dataframe with all features and returns the aggregated feature grouped by the customer id.\n", + " \n", + " df - dataframe\n", + " '''\n", + " cid = pd.Categorical(df.pop('customer_ID'), ordered=True) # get customer id\n", + " last = (cid != np.roll(cid, -1)) # mask for last statement of every customer\n", + " \n", + " df_avg = (df\n", + " .groupby(cid)\n", + " .mean()[features_avg]\n", + " .rename(columns={f: f\"{f}_avg\" for f in features_avg})\n", + " ) \n", + " \n", + " df_min = (df\n", + " .groupby(cid)\n", + " .min()[features_min]\n", + " .rename(columns={f: f\"{f}_min\" for f in features_min})\n", + " )\n", + " gc.collect()\n", + " print('Computed min')\n", + " \n", + " df_max = (df\n", + " .groupby(cid)\n", + " .max()[features_max]\n", + " .rename(columns={f: f\"{f}_max\" for f in features_max})\n", + " )\n", + " gc.collect()\n", + " print('Computed max')\n", + " \n", + " df = (df.loc[last, features_last]\n", + " .rename(columns={f: f\"{f}_last\" for f in features_last})\n", + " .set_index(np.asarray(cid[last]))\n", + " )\n", + " gc.collect()\n", + " print('Computed last')\n", + " \n", + " df_ = pd.concat([df, df_min, df_max, df_avg], axis=1, )\n", + " \n", + " del df, df_avg, df_min, df_max, cid, last\n", + " \n", + " return df_" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Computed min\n", + "Computed max\n", + "Computed last\n", + "Computed min\n", + "Computed max\n", + "Computed last\n" + ] + } + ], + "source": [ + "# apply feature engineering function\n", + "train = get_features(df_train, features_avg, features_min, features_max, features_last)\n", + "test = get_features(df_test, features_avg, features_min, features_max, features_last)" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "B_1_last False\n", + "B_2_last True\n", + "B_3_last True\n", + "B_4_last False\n", + "B_5_last False\n", + " ... \n", + "S_18_avg False\n", + "S_22_avg True\n", + "S_23_avg True\n", + "S_25_avg True\n", + "S_26_avg False\n", + "Length: 469, dtype: bool" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# check null values\n", + "train.isna().any()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Modelling: StratifiedKFold\n", + "\n", + "We cross-validate with a six-fold StratifiedKFold to handle the imbalanced nature of the target.\n", + "\n", + "Lightgbm handles null values efficiently." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "tags": [ + "block:modelling", + "prev:feature_engineering" + ] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "469 features\n", + "[20]\tvalid_0's binary_logloss: 0.267976\tvalid_0's amex_metric_score: 0.750976\n", + "Score = 0.7604229987279087\n", + "Fold 0\n", + "[20]\tvalid_0's binary_logloss: 0.267339\tvalid_0's amex_metric_score: 0.753257\n", + "Score = 0.7624180573803372\n", + "Fold 1\n", + "OOF Score: 0.76142\n" + ] + } + ], + "source": [ + "# Cross-validation\n", + "\n", + "features = [f for f in train.columns if f != 'customer_ID' and f != 'target']\n", + "\n", + "print(f\"{len(features)} features\")\n", + "\n", + "score_list = [] # lgbm score per fold\n", + "y_pred_list = [] # fold predictions list\n", + "\n", + "# init StratifiedKFold\n", + "kf = StratifiedKFold(n_splits=4)\n", + "\n", + "for fold, (idx_tr, idx_va) in enumerate(kf.split(train, target)):\n", + " \n", + " X_tr, X_va, y_tr, y_va, model = None, None, None, None, None\n", + "\n", + " X_tr = train.iloc[idx_tr][features]\n", + " X_va = train.iloc[idx_va][features]\n", + " y_tr = target[idx_tr]\n", + " y_va = target[idx_va]\n", + " \n", + " # init model\n", + " model = LGBMClassifier(n_estimators=N_EST,\n", + " learning_rate=LR, \n", + " random_state=2022)\n", + " # fit model\n", + " model.fit(X_tr, y_tr,\n", + " eval_set = [(X_va, y_va)], \n", + " eval_metric=[lgb_amex_metric],\n", + " early_stopping_rounds=30,\n", + " callbacks=[log_evaluation(20)])\n", + " \n", + " X_tr, y_tr = None, None\n", + " \n", + " # fold validation set predictions\n", + " y_va_pred = model.predict_proba(X_va, raw_score=True)\n", + " \n", + " # model score\n", + " score = amex_metric(y_va, y_va_pred)\n", + "\n", + " print(f\"Score = {score}\")\n", + " score_list.append(score)\n", + " \n", + " # test set predictions\n", + " y_pred_list.append(model.predict_proba(test[features], raw_score=True))\n", + " \n", + " print(f\"Fold {fold}\") \n", + "\n", + "# save model\n", + "joblib.dump(model, 'lgb.jl')\n", + "print(f\"OOF Score: {np.mean(score_list):.5f}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAABLMAAAI/CAYAAACMIJv7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABda0lEQVR4nO3dfbTdZXnn//eHI5LQkNBK8EcBDShjKk9RN6BQFFELNtSCpWqnWrEPGaZQfIAOsc5U7LTToA6UisoEKVQKlYqiCAp2HGkpVeQEEkIAq0AspCqVh0gkoITr98f+hh4P52Gfx713eL/Wysre9/f63vf1Ze1luq5e9/1NVSFJkiRJkiT1g+26nYAkSZIkSZLUKYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+8axuJ9Dvdtlll1q0aFG305AkSZIkSdpmrFq16gdVtXCkaxazpmjRokUMDg52Ow1JkiRJkqRtRpLvjHbNbYaSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlveGbWFK3dsJFFy6/udhqSJEmSJOkZav2Kpd1OYVb1TGdWki1JVie5Lcmnk+w4StyeSb6a5PYk65K8c5x5L0py/CTyOSLJoRO9T5IkSZIkSTOnZ4pZwOaqWlJV+wE/Bk4cJe4J4NSqejHwcuCkJC+egXyOACxmSZIkSZIk9ZBeKmYNdT3wwpEuVNV3q+rm5vMjwB3A7p1MmuSPk9zUdH+tTJJm/JSm0+vWJJ9Ksoh2Me3dTbfY4dPxUJIkSZIkSZqanjszK8mzgNcD13QQuwh4CXBjh9OfW1V/0tx7MXAM8AVgObBXVT2eZOeqejjJecCmqvrwCOsuA5YBDMxf2OHSkiRJkiRJmqpe6syam2Q1MAj8K3DBWMFJ5gGfAd5VVT/scI1XJ7kxyVrgSGDfZvxW4JIkb6W9jXFMVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqkza3NVLekkMMn2tAtZl1TVZzu8Zw7wMaBVVfcmOQOY01xeCrwS+BXgfUn2n2DukiRJkiRJmgW91JnVkeacqwuAO6rqrAncurVw9YOmq+v4Zr7tgD2r6qvA6cACYB7wCLDTtCUuSZIkSZKkKeu7YhZwGPA24MjmcPbVSX55vJuq6mHgfOA24FrgpubSAPA3zdbDW4C/bGK/ABznAfCSJEmSJEm9I1XV7Rz6WqvVqsHBwW6nIUmSJEmStM1IsqqqWiNd68fOLEmSJEmSJD1D9dIB8D8lyXOAr4xw6TVV9cAI8R+lvQVxqHOq6sKZyE+SJEmSJEmzr2eLWU3BaskE4k+auWwkSZIkSZLUC9xmKEmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqGz37NsN+sXbDRhYtv7rbaUiSJEnST1m/Ymm3U5CkGTFrnVlJtiRZnWRdkjVJTk0y6vpJDm7iVzfxxw259s4ktzVzvWucdS9Kcvwk8j0iyaETvU+SJEmSJEkzZzY7szZX1RKAJLsClwLzgfePEn8b0KqqJ5LsBqxJ8gVgMfB7wMHAj4FrklxVVd+e5nyPADYB/zzN80qSJEmSJGmSunJmVlXdDywDTk6SUWIeraonmq9zgGo+/wJw45Dr/wC8sZN1k/xxkpuarq6VW9dOckqS25PcmuRTSRYBJwLvbjrDDp/0w0qSJEmSJGnadO0A+Kq6GxgAdh0tJskhSdYBa4ETm+LVbcDhSZ6TZEfgl4E9O1z23Ko6qKr2A+YCxzTjy4GXVNUBzTrrgfOAs6tqSVVdPyyvZUkGkwxueXRjx88sSZIkSZKkqenptxlW1Y1VtS9wEPDeJHOq6g7gTODLwDXAamBLh1O+OsmNSdYCRwL7NuO3ApckeSvwxKh3/0deK6uqVVWtgR0XTOyhJEmSJEmSNGldK2Yl2Zt2Eer+8WKbAtYmYL/m+wVV9bKqeiXwEPAvHaw3B/gYcHxV7Q+cT3v7IsBS4KPAS4GbkviWR0mSJEmSpB7UlWJWkoW0t/GdW1U1SsxeW4tKSZ5P++D39c33XZu/n0f7vKxLO1h2a+HqB0nmAcc3c2wH7FlVXwVOBxYA84BHgJ0m83ySJEmSJEmaGbPZgTQ3yWpge9pb+S4Gzhoj/heB5Ul+AjwJ/H5V/aC59pkkzwF+ApxUVQ+Pt3hVPZzkfNpnbn0PuKm5NAD8TZIFQIC/bGK/AFye5FeBPxh+bpYkSZIkSZJmX0ZpjFKHWq1WDQ4OdjsNSZIkSZKkbUaSVVXVGulaTx8AL0mSJEmSJA3V9YPOkxxF++2EQ91TVcdNcJ6PAocNGz6nqi6cSn6SJEmSJEnqHV0vZlXVtcC10zDPSdOQjiRJkiRJknqY2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX2j628z7HdrN2xk0fKru52GJElSz1q/Ymm3U5AkSduQnujMSrIlyeoka5LcnOTQDu6Zn+S+JOeOE7c+yS6TyOmEJD8/0fskSZIkSZI0c3qimAVsrqolVXUg8F7gzzu4538C/ziDOZ0AWMySJEmSJEnqIb1SzBpqPvDQWAFJXgY8F/jyRCZO8rkkq5KsS7KsGRtIclGS25KsTfLuJMcDLeCSpmNs7iSfRZIkSZIkSdOoV87MmptkNTAH2A04crTAJNsB/xt4K/DaCa7z21X1YFOcuinJZ4BFwO5VtV8z/85V9XCSk4HTqmpwwk8jSZIkSZKkGdErnVlbtxkuBo4GPpkko8T+PvDFqrpvEuuckmQN8HVgT2Af4G5g7yQfSXI08MPxJkmyLMlgksEtj26cRBqSJEmSJEmajF7pzHpKVX2tObB9IXD/CCGvAA5P8vvAPODZSTZV1fKx5k1yBO1OrldU1aNJrgPmVNVDSQ4EjgJOBN4E/PY4Oa4EVgLssNs+NYHHkyRJkiRJ0hT0XDEryWJgAHhgpOtV9ZtDYk8AWuMVshoLgIeaQtZi4OXNHLsAP66qzyT5JvA3TfwjwE6TfhBJkiRJkiRNu14pZm09MwsgwNurass0r3ENcGKSO4Bv0t5qCLA7cGFzFhe036YIcBFwXpLNtLu5Nk9zPpIkSZIkSZqgVLlLbiparVYNDnpGvCRJkiRJ0nRJsqqqWiNd65UD4CVJkiRJkqRx9co2w6dJsj9w8bDhx6vqkFHibwR2GDb8tqpaOxP5SZIkSZIkafb1bDGrKUItmUD8iEUuSZIkSZIkbTvcZihJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6hs9+zbDfrF2w0YWLb+622lIkrRNW79iabdTkCRJUo+wM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOk78NUkeTnJVB3Nfl6Q1iZyOTfLiid4nSZIkSZKkmdMTxSxgc1UtqaoDgfcCfz5O/IeAt81wTscCFrMkSZIkSZJ6SK8Us4aaDzw0VkBVfQV4ZKITJ/l4ksEk65J8YMj4iiS3J7k1yYebzrA3AB9qOsZeMNG1JEmSJEmSNP165QD4uUlWA3OA3YAjZ2id91XVg0kGgK8kOQDYABwHLK6qSrJzVT2c5Ergqqq6fPgkSZYBywAG5i+coVQlSZIkSZI0XK90Zm3dZrgYOBr4ZJLMwDpvSnIzcAuwL+1thBuBx4ALkrwReHS8SapqZVW1qqo1sOOCGUhTkiRJkiRJI+mVYtZTquprwC7AtLY8JdkLOA14TVUdAFwNzKmqJ4CDgcuBY4BrpnNdSZIkSZIkTZ9e2Wb4lCSLgQHggWmeej7wI2BjkucCrweuSzIP2LGqvpjkBuDuJv4RYKdpzkGSJEmSJElT0CvFrK1nZgEEeHtVbRktOMn1wGJgXpL7gN+pqmvHWqCq1iS5BbgTuBe4obm0E/D5JHOatd/TjH8KOD/JKcDxVXXXSPPuv/sCBlcs7eQZJUmSJEmSNEU9UcyqqoEJxh8+gdgjhnw+YZSwg0e47wbaZ2pJkiRJkiSpR/TcmVmSJEmSJEnSaHqiM2skSfYHLh42/HhVHTJK/BXAXsOGTx9v+6EkSZIkSZL6R88Ws6pqLbBkAvHHzVw2kiRJkiRJ6gVuM5QkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6Rs8eAN8v1m7YyKLlV3c7DUmSZtX6FUu7nYIkSZKeoezMkiRJkiRJUt/omWJWkvclWZfk1iSrkxwyStwFSdY0cZcnmTfbuUqSJEmSJKk7eqKYleQVwDHAS6vqAOC1wL2jhL+7qg5s4v4VOHmW0pQkSZIkSVKX9UQxC9gN+EFVPQ5QVT+oqn8bKbCqfgiQJMBcoEabNMmvJLkxyS1J/m+S5ybZLsn6JDsPiftWc+0FSb6eZG2SP02yaTofUpIkSZIkSVPTK8WsLwN7JvmXJB9L8qqxgpNcCHwPWAx8ZIzQfwJeXlUvAT4F/LeqehL4PHBcM9chwHeq6vvAOcA5VbU/cN8Y6y9LMphkcMujGzt/SkmSJEmSJE1JTxSzqmoT8DJgGfDvwGVJThgj/h3AzwN3AG8eY+o9gGuTrAX+ENi3Gb9syH1vab4DvAL4dPP50jHWX1lVrapqDey4YIzlJUmSJEmSNJ16opgFUFVbquq6qno/7XOwfm28eNrdVmPFfQQ4t+m0+i/AnGb8a8ALkywEjgU+O8X0JUmSJEmSNAt6opiV5EVJ9hkytAT4zghxSfLCrZ+BNwB3jjH1AmBD8/ntWwerqoArgLOAO6rqgebS1/mP4thbJv4kkiRJkiRJmknP6nYCjXnAR5pD2Z8Avk17y+FwAf46yfzm8xrgv44x7xnAp5M8BPw/YK8h1y4DbgJOGDL2LuBvkrwPuAYY90Cs/XdfwOCKpeOFSZIkSZIkaRr0RDGrqlYBh3YQ9yRw2ATm/Tztw95HujZIuyA21AbaB8ZXkrcAL+p0LUmSJEmSJM28nihm9ZCXAec2WxgfBn67u+lIkiRJkiRpqJ4tZiW5gp/eFghwelVdO0Ls+4BfHzb86ar6s4msWVXXAwdOKFFJkiRJkiTNmp4tZlXVcROI/TNgQoUrSZIkSZIk9Z+eeJuhJEmSJEmS1AmLWZIkSZIkSeobFrMkSZIkSZLUNyxmSZIkSZIkqW/07AHw/WLtho0sWn51t9OQJPWY9SuWdjsFSZIkaZtkZ5YkSZIkSZL6Rk8Us5JsSbI6yZokNyc5dIzY5zcxq5OsS3LiOHOvT7LLJHI6IcnPT/Q+SZIkSZIkzZxe2Wa4uaqWACQ5Cvhz4FWjxH4XeEVVPZ5kHnBbkiur6t+mOacTgNuA6Z5XkiRJkiRJk9Qrxayh5gMPjXaxqn485OsOTKC7LMnngD2BOcA5VbUyyQBwAdACCvgr4N7m+yVJNtMunm2e4HNIkiRJkiRpmvVKMWtuktW0i0y7AUeOFZxkT+Bq4IXAH06gK+u3q+rBJHOBm5J8BlgE7F5V+zVz71xVDyc5GTitqgZHWH8ZsAxgYP7CDpeWJEmSJEnSVPXEmVk02wyrajFwNPDJJBktuKruraoDaBez3p7kuR2uc0qSNcDXaXdo7QPcDeyd5CNJjgZ+ON4kVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqWY9ZSq+hqwCzBuy1PTkXUbcPh4sUmOAF5Le8vggcAtwJyqegg4ELgOOBH4xCRTlyRJkiRJ0gzruWJWksXAAPDAKNf3aLYJkuRngV8EvtnB1AuAh6rq0WaNlzdz7AJsV1WfAf478NIm/hFgp6k8iyRJkiRJkqZXr52ZBRDg7VW1ZZTYXwD+d5JqYj9cVWs7WOMa4MQkd9Aufn29Gd8duDDJ1sLee5u/LwLOG+8A+P13X8DgiqUdLC9JkiRJkqSpSlV1O4e+1mq1anDwaWfES5IkSZIkaZKSrKqq1kjXem6boSRJkiRJkjSaXtlm+DRJ9gcuHjb8eFUdMkr8jcAOw4bf1uEWREmSJEmSJPWBni1mNUWoJROIH7HIJUmSJEmSpG2H2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvtGzB8D3i7UbNrJo+dXdTkOSnjHWr1ja7RQkSZIkdZGdWZIkSZIkSeobs1LMSrIlyeok65KsSXJqknHXTvK8JJuSnNZ83zPJV5Pc3sz1zpnPXpIkSZIkSb1itrYZbq6qJQBJdgUuBeYD7x/nvrOALw35/gRwalXdnGQnYFWSv6+q22cgZ0mSJEmSJPWYWd9mWFX3A8uAk5NktLgkxwL3AOuG3Pvdqrq5+fwIcAew+xhz/F6Sm5pusM8k2THJgiTf2doZluRnktybZPskByW5teki+1CS26bloSVJkiRJkjQtunJmVlXdDQwAu450Pck84HTgA6PNkWQR8BLgxjGW+mxVHVRVB9IufP1OVW0EVgOvamKOAa6tqp8AFwL/peki2zLG2suSDCYZ3PLoxjGWlyRJkiRJ0nTq1QPgzwDOrqpNI11sil2fAd5VVT8cY579klyfZC3wm8C+zfhlwJubz28BLkuyM7BTVX2tGb90tEmramVVtaqqNbDjgk6fSZIkSZIkSVM0W2dm/ZQke9PufLp/lJBDgOOTfBDYGXgyyWNVdW6S7WkXsi6pqs+Os9RFwLFVtSbJCcARzfiVwP9K8nPAy4D/B+w0+SeSJEmSJEnSbJj1YlaShcB5wLlVVSPFVNXhQ+LPADY1hawAFwB3VNVZHSy3E/DdpgD2m8CGZv5NSW4CzgGuqqotwMNJHklySFXdSLtjS5IkSZIkST1ktopZc5OsBran/UbCi2m/qXCiDgPeBqxt5gP4o6r64ijx/4P2mVr/3vw9tPvqMuDT/Ee3FsDvAOcneRL4B8ADsSRJkiRJknpIRmmOekZKMm/rOV1JlgO7VdU7x7qn1WrV4ODgrOQnSZIkSZL0TJBkVVW1RrrWlTOzetjSJO+l/d/lO8AJ3U1HkiRJkiRJQ3W1mJXkKODMYcP3VNVxE5zno7S3IA51TlVdOJF5quoy2tsPJUmSJEmS1IO6WsyqqmuBa6dhnpOmIR1JkiRJkiT1uO26nYAkSZIkSZLUKYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt/o6tsMtwVrN2xk0fKru52GJHXN+hVLu52CJEmSpGeQnunMSvK+JOuS3JpkdZJDRom7IMmaJu7yJPPGmPOMJKdNIpclSX55ovdJkiRJkiRpZvVEMSvJK4BjgJdW1QHAa4F7Rwl/d1Ud2MT9K3DyDKS0BLCYJUmSJEmS1GN6opgF7Ab8oKoeB6iqH1TVv40UWFU/BEgSYC5QnSyQ5PeS3NR0dX0myY7N+K8nua0Z/8ckzwb+BHhz0yH25ml4PkmSJEmSJE2DXilmfRnYM8m/JPlYkleNFZzkQuB7wGLgIx2u8dmqOqiqDgTuAH6nGf9j4Khm/A1V9eNm7LKqWlJVl42w/rIkg0kGtzy6scPlJUmSJEmSNFU9Ucyqqk3Ay4BlwL8DlyU5YYz4dwA/T7so1Wnn1H5Jrk+yFvhNYN9m/AbgoiS/Bwx0mO/KqmpVVWtgxwUdLi9JkiRJkqSp6oliFkBVbamq66rq/bTPwfq18eKBT40XN8RFwMlVtT/wAWBOM8+JwH8H9gRWJXnO5J5AkiRJkiRJM60nillJXpRknyFDS4DvjBCXJC/c+hl4A3Bnh8vsBHw3yfa0O7O2zvmCqrqxqv6YdlfYnsAjTbwkSZIkSZJ6yLO6nUBjHvCRJDsDTwDfpr3lcLgAf51kfvN5DfBfO1zjfwA30i5Y3ch/FKs+1BTSAnylmfNfgeVJVgN/PtK5WZIkSZIkSZp9qeroZYAaRavVqsHBwW6nIUmSJEmStM1IsqqqWiNd64lthpIkSZIkSVInemWb4dMkuQLYa9jw6VV17Qix7wN+fdjwp6vqz2YqP0mSJEmSJM2+ni1mVdVxE4j9M8DClSRJkiRJ0jbObYaSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL7Rs28z7BdrN2xk0fKru52GJE3Z+hVLu52CJEmSJI1r1jqzkmxJsjrJuiRrkpyaZNz1kzwvyaYkpw0Z+6sk9ye5rYP7L0py/CTyPSLJoRO9T5IkSZIkSTNnNrcZbq6qJVW1L/A64PXA+zu47yzgS8PGLgKOnt70nuYIwGKWJEmSJElSD+nKmVlVdT+wDDg5SUaLS3IscA+wbtj9/wg8ONF1k/xxkpuS3JZk5da1k5yS5PYktyb5VJJFwInAu5tussMnupYkSZIkSZKmX9cOgK+qu4EBYNeRrieZB5wOfGAalz23qg6qqv2AucAxzfhy4CVVdQBwYlWtB84Dzm66ya4fltuyJINJBrc8unEa05MkSZIkSdJYevlthmfQLiZtmsY5X53kxiRrgSOBfZvxW4FLkrwVeGK8SapqZVW1qqo1sOOCaUxPkiRJkiRJY+na2wyT7A1sAe4fJeQQ4PgkHwR2Bp5M8lhVnTvJ9eYAHwNaVXVvkjOAOc3lpcArgV8B3pdk/8msIUmSJEmSpJnVlWJWkoW0t/GdW1U1UkxVHT4k/gxg02QLWY2thasfNFsYjwcub96ouGdVfTXJPwFvAeYBjwDzp7CeJEmSJEmSptlsbjOc2xymvg74v8CXmeR5WEn+Fvga8KIk9yX5nfHuqaqHgfOB24BrgZuaSwPA3zRbD28B/rKJ/QJwnAfAS5IkSZIk9Y6M0hilDrVarRocHOx2GpIkSZIkSduMJKuqqjXStV4+AF6SJEmSJEn6KV07AH6rJEcBZw4bvqeqjpvgPB8FDhs2fE5VXTiV/CRJkiRJktQ7ul7MqqpraZ9hNdV5TpqGdCRJkiRJktTD3GYoSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobXX+bYb9bu2Eji5Zf3e00JGnK1q9Y2u0UJEmSJGlcs9aZlWRLktVJ1iVZk+TUJOOun+R5STYlOa35PifJN5o51iX5wDj3X5ekNYl8j03y4oneJ0mSJEmSpJkzm9sMN1fVkqraF3gd8Hrg/R3cdxbwpSHfHweOrKoDgSXA0UlePt3JAscCFrMkSZIkSZJ6SFfOzKqq+4FlwMlJMlpckmOBe4B1Q+6tqtrUfN2++VOdrJvk40kGh3d0JVmR5PYktyb5cJJDgTcAH2q6yV4w0WeUJEmSJEnS9OvamVlVdXeSAWBX4PvDryeZB5xOu4vrtGHXBoBVwAuBj1bVjR0u+76qerC5/ytJDgA2AMcBi6uqkuxcVQ8nuRK4qqouHyG3ZbSLcQzMX9jh0pIkSZIkSZqqXn6b4RnA2UO6sJ5SVVuqagmwB3Bwkv06nPNNSW4GbgH2pb2NcCPwGHBBkjcCj443SVWtrKpWVbUGdlzQ4dKSJEmSJEmaqq51ZiXZG9gC3D9KyCHA8Uk+COwMPJnksao6d2tA00H1VeBo4LZx1tuLdofXQVX1UJKLgDlV9USSg4HXAMcDJwNHTunhJEmSJEmSNCO6UsxKshA4Dzi3qkY876qqDh8SfwawqarObe79SVPImkt7G+KZHSw7H/gRsDHJc2kfQH9ds51xx6r6YpIbgLub+EeAnSb3hJIkSZIkSZoJs1nMmptkNe0D258ALqb9psKJ2g346+bcq+2Av6uqq8a7qarWJLkFuBO4F7ihubQT8Pkkc4AA72nGPwWcn+QU4PiqumsSuUqSJEmSJGkaZZTGKHWo1WrV4OBgt9OQJEmSJEnaZiRZVVWtka718gHwkiRJkiRJ0k/p2gHwWyU5iqefeXVPVR03wXmuAPYaNnx6VV07lfwkSZIkSZLUO7pezGqKTVMuOE20+CVJkiRJkqT+4zZDSZIkSZIk9Q2LWZIkSZIkSeobFrMkSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLf6PrbDPvd2g0bWbT86m6nIUlPWb9iabdTkCRJkqQZM2udWUm2JFmdZF2SNUlOTTLu+kmel2RTktOa73OSfKOZY12SD8x89pIkSZIkSeoFs9mZtbmqlgAk2RW4FJgPvH+c+84CvjTk++PAkVW1Kcn2wD8l+VJVfX0GcpYkSZIkSVIP6cqZWVV1P7AMODlJRotLcixwD7BuyL1VVZuar9s3f2qMOf44yU1JbkuyMm2Lk3xjSMyiJGubz7+c5M4kq5L8ZZKrpvKskiRJkiRJmj5dOwC+qu4GBoBdR7qeZB5wOvC0bYRJBpKsBu4H/r6qbhxjqXOr6qCq2g+YCxxTVXcCz06yVxPzZuCyJHOA/wO8vqpeBiwcJbdlSQaTDG55dGMnjytJkiRJkqRp0MtvMzwDOHtIF9ZTqmpLs2VxD+DgJPuNMc+rk9zYdF4dCezbjP8d7SIWzd+XAYuBu6vqnmb8b0easKpWVlWrqloDOy6Y4GNJkiRJkiRpsrr2NsMkewNbaHdXjeQQ4PgkHwR2Bp5M8lhVnbs1oKoeTvJV4GjgthHWmAN8DGhV1b1JzgDmNJcvAz6d5LPtqepbSZZMy8NJkiRJkiRpRnSlMyvJQuA82lsARzzvqqoOr6pFVbUI+Avgf1XVuUkWJtm5mWcu8DrgzlGW2lq4+kGzbfH4IfPfRbuY9j9oF7YAvgnsnWRR831r55YkSZIkSZJ6wGx2Zs1tzrnaHngCuJj2mwonajfgr5MM0C7G/V1VjXhIe9O5dT7trq3vATcNC7kM+BCwVxO/OcnvA9ck+dEI8ZIkSZIkSeqijNIY9YyVZF5VbWresvhR4FtVdfZo8a1WqwYHB2cvQUmSJEmSpG1cklVV1RrpWi8fAN8tv9d0kK0DFtB+u6EkSZIkSZJ6QNcOgN8qyVHAmcOG76mq4yY4zxU02wWHOL2qrp3IPE0X1qidWJIkSZIkSeqerhezmmLThApOo8wzoeKXJEmSJEmS+o/bDCVJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfaPrbzPsd2s3bGTR8qu7nYYkPWX9iqXdTkGSJEmSZsysdWYl2ZJkdZJ1SdYkOTXJuOsneV6STUlOGzL2ziS3NXO9a5z7L0py/CTyPSLJoRO9T5IkSZIkSTNnNjuzNlfVEoAkuwKXAvOB949z31nAl7Z+SbIf8HvAwcCPgWuSXFVV357mfI8ANgH/PM3zSpIkSZIkaZK6cmZWVd0PLANOTpLR4pIcC9wDrBsy/AvAjVX1aFU9AfwD8MZO1k3yx0luarq6Vm5dO8kpSW5PcmuSTyVZBJwIvLvpJjt8Ms8pSZIkSZKk6dW1A+Cr6m5gANh1pOtJ5gGnAx8Yduk24PAkz0myI/DLwJ4dLntuVR1UVfsBc4FjmvHlwEuq6gDgxKpaD5wHnF1VS6rq+gk8miRJkiRJkmZIL7/N8AzaxaRNQwer6g7gTODLwDXAamBLh3O+OsmNSdYCRwL7NuO3ApckeSvwxHiTJFmWZDDJ4JZHN3a4tCRJkiRJkqaqa8WsJHvTLkLdP0rIIcAHk6wH3gX8UZKTAarqgqp6WVW9EngI+JcO1psDfAw4vqr2B84H5jSXlwIfBV4K3JRkzLPEqmplVbWqqjWw44LxlpYkSZIkSdI0mc0D4J+SZCHtbXznVlWNFFNVhw+JPwPYVFXnNt93rar7kzyP9nlZL+9g2a2Fqx80WxiPBy5v3qi4Z1V9Nck/AW8B5gGP0D6gXpIkSZIkST1iNotZc5OsBranvZXvYtpvKpyMzyR5DvAT4KSqeni8G6rq4STn0z5z63vATc2lAeBvkiwAAvxlE/sF2sWuXwX+wHOzJEmSJEmSui+jNEapQ61WqwYHB7udhiRJkiRJ0jYjyaqqao10rZcPgJckSZIkSZJ+SlfOzBoqyVG030441D1VddwE5/kocNiw4XOq6sKp5CdJkiRJkqTe0fViVlVdC1w7DfOcNA3pSJIkSZIkqYe5zVCSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1De6/jbDfrd2w0YWLb+622lI6mHrVyztdgqSJEmStM2wM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOkbskiRfS7Iuya1J3jzO3NclaU0ip2OTvHii90mSJEmSJGnm9EQxC9hcVUuq6kDgvcCfjxH7KPBbVbUvcDTwF0l2noGcjgUsZkmSJEmSJPWQXilmDTUfeGi0i1X1L1X1rebzvwH3Aws7mTjJx5MMNl1dHxgyviLJ7U2n14ebzrA3AB9qOsZeMKUnkiRJkiRJ0rTolQPg5yZZDcwBdgOO7OSmJAcDzwbu6nCd91XVg0kGgK8kOQDYABwHLK6qSrJzVT2c5Ergqqq6fIR1lwHLAAbmd1RHkyRJkiRJ0jTolc6srdsMF9PeOvjJJBnrhiS7ARcD76iqJztc501JbgZuAfalvY1wI/AYcEGSN9LexjimqlpZVa2qag3suKDDpSVJkiRJkjRVvVLMekpVfQ3YhTG2DiaZD1xNu9Pq653Mm2Qv4DTgNVV1QHP/nKp6AjgYuBw4Brhmak8gSZIkSZKkmdIr2wyfkmQxMAA8MMr1ZwNXAJ8caQvgGOYDPwI2Jnku8HrguiTzgB2r6otJbgDubuIfAXaa5GNIkiRJkiRpBvRKMWvrmVkAAd5eVVtGiX0T8ErgOUlOaMZOqKrVo8QDUFVrktwC3AncC9zQXNoJ+HySOc3a72nGPwWcn+QU4PiqGvFcrv13X8DgiqXjPJ4kSZIkSZKmQ6qq2zn0tVarVYODg91OQ5IkSZIkaZuRZFVVtUa61nNnZkmSJEmSJEmj6ZVthk+TZH/abysc6vGqOmSU+CuAvYYNn15V185EfpIkSZIkSZp9PVvMqqq1wJIJxB83c9lIkiRJkiSpF7jNUJIkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobPXsAfL9Yu2Eji5Zf3e00JPWw9SuWdjsFSZIkSdpm2JklSZIkSZKkvjFrxawkW5KsTrIuyZokpyYZd/0kz0uyKclpQ8Z2TnJ5kjuT3JHkFWPcf1GS4yeR7xFJDp3ofZIkSZIkSZo5s7nNcHNVLQFIsitwKTAfeP84950FfGnY2DnANVV1fJJnAztOc64ARwCbgH+egbklSZIkSZI0CV3ZZlhV9wPLgJOTZLS4JMcC9wDrhowtAF4JXNDM9eOqeriTdZP8cZKbktyWZOXWtZOckuT2JLcm+VSSRcCJwLubbrLDJ/WgkiRJkiRJmlZdOzOrqu4GBoBdR7qeZB5wOvCBYZf2Av4duDDJLUk+keRnOlz23Ko6qKr2A+YCxzTjy4GXVNUBwIlVtR44Dzi7qpZU1fXDcluWZDDJ4JZHN3a4tCRJkiRJkqaqlw+AP4N2MWnTsPFnAS8FPl5VLwF+RLsY1YlXJ7kxyVrgSGDfZvxW4JIkbwWeGG+SqlpZVa2qag3suKDDpSVJkiRJkjRVs3lm1k9JsjewBbh/lJBDgOOTfBDYGXgyyWPA5cB9VXVjE3c5HRSzkswBPga0qureJGcAc5rLS2lvXfwV4H1J9p/UQ0mSJEmSJGlGdaWYlWQh7W1851ZVjRRTVYcPiT8D2FRV5zbf703yoqr6JvAa4PYOlt1auPpBs4XxeODy5o2Ke1bVV5P8E/AWYB7wCO0D6iVJkiRJktQjZrOYNTfJamB72lv5Lqb9psLJ+APa2wKfDdwNvGO8G6rq4STnA7cB3wNuai4NAH/THCwf4C+b2C/QLnb9KvAHw8/N2mr/3RcwuGLpJB9DkiRJkiRJE5FRGqPUoVarVYODg91OQ5IkSZIkaZuRZFVVtUa61ssHwEuSJEmSJEk/pWsHwG+V5CjgzGHD91TVcROc56PAYcOGz6mqC6eSnyRJkiRJknpH14tZVXUtcO00zHPSNKQjSZIkSZKkHuY2Q0mSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvdP0A+H63dsNGFi2/uttpSJoF61cs7XYKkiRJkvSMZ2eWJEmSJEmS+kZPFLOSbEmyOsltSb6QZOdx4q9J8nCSqzqY+7okrUnkdGySF0/0PkmSJEmSJM2cnihmAZuraklV7Qc8CJw0TvyHgLfNcE7HAhazJEmSJEmSekivFLOG+hqw+1gBVfUV4JGJTpzk40kGk6xL8oEh4yuS3J7k1iQfTnIo8AbgQ03H2AsmupYkSZIkSZKmX08dAJ9kAHgNcMEMLfG+qnqwWecrSQ4ANgDHAYurqpLsXFUPJ7kSuKqqLh8hz2XAMoCB+QtnKFVJkiRJkiQN1yudWXOTrAa+BzwX+PsZWudNSW4GbgH2pb2NcCPwGHBBkjcCj443SVWtrKpWVbUGdlwwQ6lKkiRJkiRpuF4pZm2uqiXA84Ew/plZE5ZkL+A04DVVdQBwNTCnqp4ADgYuB44BrpnutSVJkiRJkjQ9eqWYBUBVPQqcApyaZLq3QM4HfgRsTPJc4PUASeYBC6rqi8C7gQOb+EeAnaY5B0mSJEmSJE3BmAWjJO8Z63pVnTW96UBV3ZLkVuA3gItHyet6YDEwL8l9wO9U1bXjzLsmyS3AncC9wA3NpZ2AzyeZQ7srbOszfwo4P8kpwPFVdddI8+6/+wIGVyyd0DNKkiRJkiRpcsbrfpqVzqSqmjfs+6+ME3/4BOY+YsjnE0YJO3iE+26gfaaWJEmSJEmSesSYxayq+sBsJSJJkiRJkiSNp6NzqZLsAXwEOKwZuh54Z1XdN1OJJdmfp28zfLyqDhkl/gpgr2HDp4+3/VCSJEmSJEn9o9ND1i8ELgV+vfn+1mbsdTORFEBVrQWWTCD+uJnKRZIkSZIkSb2h07cZLqyqC6vqiebPRcDCGcxLkiRJkiRJeppOi1kPJHlrkoHmz1uBB2YyMUmSJEmSJGm4TotZvw28Cfge8F3geOCEGcpJkiRJkiRJGlGnZ2b9CfD2qnoIIMnPAR+mXeSSJEmSJEmSZkWnnVkHbC1kAVTVg8BLZiYlSZIkSZIkaWSddmZtl+Rnh3VmdXrvNm3tho0sWn51t9OQnpHWr1ja7RQkSZIkSbOs086s/w18Lcn/TPI/gX8GPjiRhZJsSbI6yZokNyc5dJz4a5I8nOSqYeMnJ/l2kkqyyzhznJDk3Ink2dy3KMl/nuh9kiRJkiRJmlkdFbOq6pPAG4HvN3/eWFUXT3CtzVW1pKoOBN4L/Pk48R8C3jbC+A3Aa4HvTHD9iVgEWMySJEmSJEnqMR1vFayq24Hbp2nd+cBDYwVU1VeSHDHC+C0ASSa0YJJfAf478GzgAeA3q+r7SV4FnLN1euCVwArgF5KsBv66qs6e0GKSJEmSJEmaEbN57tXcpjg0B9gNOHIW1wb4J+DlVVVJfhf4b8CpwGnASVV1Q5J5wGPAcuC0qjpmpImSLAOWAQzMXzgryUuSJEmSJGl2i1mbq2oJQJJXAJ9Msl9V1SytvwdwWZLdaHdn3dOM3wCcleQS4LNVdd94XV9VtRJYCbDDbvvMVv6SJEmSJEnPeJ0eAD+tquprwC7AbLY1fQQ4t6r2B/4L7Q4xqmoF8LvAXOCGJItnMSdJkiRJkiRNwGx2Zj2lKRgN0D67arYsADY0n98+JJcXVNVaYG2Sg4DFwL3ATrOYmyRJkiRJkjowm51Zc5Osbs7Nugx4e1VtGS04yfXAp4HXJLkvyVHN+ClJ7qO9bfDWJJ/ocP0zgE8nWQX8YMj4u5LcluRW4CfAl4BbgS1J1iR598QeU5IkSZIkSTMls3dk1bap1WrV4OBgt9OQJEmSJEnaZiRZVVWtka515cwsSZIkSZIkaTK6cmbWVkn2By4eNvx4VR0ywXneAbxz2PANVXXSVPKTJEmSJElSb+lqMas5eH3JNMxzIXDhlBOSJEmSJElST3OboSRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb3T1bYbbgrUbNrJo+dXdTkN6Rlq/Ymm3U5AkSZIkzbJZ6cxKsiXJ6iTrkqxJcmqScddO8rwkm5Kc1nzfM8lXk9zezPXOacrvn6djHkmSJEmSJM2s2erM2lxVSwCS7ApcCswH3j/OfWcBXxry/Qng1Kq6OclOwKokf19Vt08luao6dCr3S5IkSZIkaXbM+plZVXU/sAw4OUlGi0tyLHAPsG7Ivd+tqpubz48AdwC7jzHHdUnOTjKY5I4kByX5bJJvJfnTIXGbmr+PaO65PMmdSS4ZK0dJkiRJkiTNrq4cAF9VdwMDwK4jXU8yDzgd+MBocyRZBLwEuHGc5X5cVS3gPODzwEnAfsAJSZ4zQvxLgHcBLwb2Bg4bYe1lTYFscMujG8dZXpIkSZIkSdOlV99meAZwdlVtGuliU+z6DPCuqvrhOHNd2fy9FljXdHc9DtwN7DlC/Deq6r6qehJYDSwaHlBVK6uqVVWtgR0XdPI8kiRJkiRJmgZdeZthkr2BLcD9o4QcAhyf5IPAzsCTSR6rqnOTbE+7kHVJVX22g+Ueb/5+csjnrd9Hev6hMVtGiZEkSZIkSVIXzHqhJslC2lv+zq2qGimmqg4fEn8GsKkpZAW4ALijqs6ajXwlSZIkSZLUO2armDU3yWpge9pvJLyY9psKJ+ow4G3A2mY+gD+qqi9OR5KSJEmSJEnqbRmlOUodarVaNTg42O00JEmSJEmSthlJVjUv9HuaXj0AXpIkSZIkSXqarh5unuQo4Mxhw/dU1XETnOejtLcgDnVOVV04lfwkSZIkSZLUW7pazKqqa4Frp2Gek6YhHUmSJEmSJPU4txlKkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvpGV99muC1Yu2Eji5Zf3e00pJ6wfsXSbqcgSZIkSdrG9URnVpItSVYnWZPk5iSHdhi/OsmV48Rel6Q1iZyOTfLiid4nSZIkSZKkmdMrnVmbq2oJQJKjgD8HXtVJ/Aw6FrgKuH2G15EkSZIkSVKHeqIza5j5wEMzMXGSjycZTLIuyQeGjK9IcnuSW5N8uOkMewPwoab76wUzkY8kSZIkSZImplc6s+YmWQ3MAXYDjhwnfk6SQeAJYEVVfa7Ddd5XVQ8mGQC+kuQAYANwHLC4qirJzlX1cLN98aqqunz4JEmWAcsABuYv7HBpSZIkSZIkTVWvdGZtrqolVbUYOBr4ZJKMEf/8qmoB/xn4iwl0Tr0pyc3ALcC+wIuBjcBjwAVJ3gg8Ot4kVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqWY9ZSq+hqwCzBqy1NVbWj+vhu4DnjJePMm2Qs4DXhNVR0AXA3MqaongIOBy4FjgGum+AiSJEmSJEmaIT1XzEqyGBgAHhjl+s8m2aH5vAtwGJ0d0j4f+BGwMclzgdc3c8wDFlTVF4F3Awc28Y8AO03hUSRJkiRJkjTNeu3MLIAAb6+qLaPE/gLwf5I8SbsYt6Kqxi1mVdWaJLcAdwL3Ajc0l3YCPp9kTrP2e5rxTwHnJzkFOL6q7prEc0mSJEmSJGkapaq6nUNfa7VaNTg42O00JEmSJEmSthlJVjXnpT9Nz20zlCRJkiRJkkbTK9sMnybJ/sDFw4Yfr6pDRom/Athr2PDpVXXtTOQnSZIkSZKk2dezxayqWgssmUD8cTOXjSRJkiRJknqB2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX2jZ99m2C/WbtjIouVXdzsNqSesX7G02ylIkiRJkrZxM9aZlWRLktVJ1iVZk+TUJKOul+TgJn51E3/ckGvvTHJbM9e7ZipnSZIkSZIk9baZ7MzaXFVLAJLsClwKzAfeP0r8bUCrqp5IshuwJskXgMXA7wEHAz8GrklyVVV9ewZzlyRJkiRJUg+alTOzqup+YBlwcpKMEvNoVT3RfJ0DVPP5F4Abh1z/B+CNo62V5LokZycZTHJHkoOSfDbJt5L86ZC4zyVZ1XR7LWvGnt/E7ZJkuyTXJ/mlqf8XkCRJkiRJ0nSYtTOzquruJAPArsD3R4pJcgjwV8Dzgbc1XVq3AX+W5DnAZuCXgcFxlvtxVbWSvBP4PPAy4EHgriRnV9UDwG9X1YNJ5gI3JflMVX0nyZnAx4FvALdX1ZdHyHMZ7eIcA/MXTvQ/hSRJkiRJkiapp95mWFU3VtW+wEHAe5PMqao7gDOBLwPXAKuBLeNMdWXz91pgXVV9t6oeB+4G9myunZJkDfD1ZmyfJodP0N4OeSJw2ih5rqyqVlW1BnZcMLmHlSRJkiRJ0oTNWjEryd60i1D3jxfbFLA2Afs13y+oqpdV1SuBh4B/GWeKx5u/nxzyeev3ZyU5Angt8IqqOhC4hfbWRpLsCOzRxM8b98EkSZIkSZI0a2almJVkIXAecG5V1SgxeyV5VvP5+bQPfl/ffN+1+ft5tM/LunSKKS0AHqqqR5MsBl4+5NqZwCXAHwPnT3EdSZIkSZIkTaOZPDNrbpLVwPbAE8DFwFljxP8isDzJT2h3UP1+Vf2gufaZ5sysnwAnVdXDU8ztGuDEJHcA36S91ZAkr6K9xfGwqtqS5NeSvKOqLpziepIkSZIkSZoGGaVRSh1qtVo1ODjeefSSJEmSJEnqVJJVVdUa6VpPHQAvSZIkSZIkjWUmtxmOKMlRtM+lGuqeqjpugvN8FDhs2PA5bgmUJEmSJEnads16MauqrgWunYZ5TpqGdCRJkiRJktRH3GYoSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobs/42w23N2g0bWbT86m6nIfWE9SuWdjsFSZIkSdI2ric6s5JsSbI6yW1JvpBk5zFilyT5WpJ1SW5N8uZx5r4uSWsSOR2b5MUTvU+SJEmSJEkzpyeKWcDmqlpSVfsBDwInjRH7KPBbVbUvcDTwF2MVv6bgWMBiliRJkiRJUg/plWLWUF8Ddh/tYlX9S1V9q/n8b8D9wMJOJk7y8SSDTVfXB4aMr0hye9Pp9eEkhwJvAD7UdIy9YEpPJEmSJEmSpGnRU2dmJRkAXgNc0GH8wcCzgbs6XOJ9VfVgs85XkhwAbACOAxZXVSXZuaoeTnIlcFVVXT7xJ5EkSZIkSdJM6JXOrLlJVgPfA54L/P14NyTZDbgYeEdVPdnhOm9KcjNwC7Av7W2EG4HHgAuSvJH2Nsbx1l7WdHgNbnl0Y4dLS5IkSZIkaap6pZi1uaqWAM8HwthnZpFkPnA17U6rr3eyQJK9gNOA11TVAc39c6rqCeBg4HLgGOCa8eaqqpVV1aqq1sCOCzpZXpIkSZIkSdOgV4pZAFTVo8ApwKlJRtwCmeTZwBXAJye4BXA+8CNgY5LnAq9v5psHLKiqLwLvBg5s4h8BdprUg0iSJEmSJGlG9FQxC6CqbgFuBX5jlJA3Aa8ETmgOZ1+dZEkH866hvb3wTuBS4Ibm0k7AVUluBf4JeE8z/ingD5Pc4gHwkiRJkiRJvSFV1e0c+lqr1arBwcFupyFJkiRJkrTNSLKqqlojXeu5zixJkiRJkiRpNCOeS9ULkuxP+22FQz1eVYeMEn8FsNew4dOr6tqZyE+SJEmSJEmzr2eLWVW1FlgygfjjZi4bSZIkSZIk9QK3GUqSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+kbPvs2wX6zdsJFFy6/udhrStFq/Ymm3U5AkSZIkaUR2ZkmSJEmSJKlvzFgxK8mWJKuTrEuyJsmpScZdL8nzkmxKclrzfc8kX01yezPXO2cqZ0mSJEmSJPW2mdxmuLmqlgAk2RW4FJgPvH+c+84CvjTk+xPAqVV1c5KdgFVJ/r6qbp+BnCVJkiRJktTDZmWbYVXdDywDTk6S0eKSHAvcA6wbcu93q+rm5vMjwB3A7mPMcV2Ss5MMJrkjyUFJPpvkW0n+dEjc55Ksarq9ljVjz2/idkmyXZLrk/zSFB9fkiRJkiRJ02TWDoCvqruTDAC7At8ffj3JPOB04HXAaSPNkWQR8BLgxnGW+3FVtZotiZ8HXgY8CNyV5OyqegD47ap6MMlc4KYkn6mq7yQ5E/g48A3g9qr68gh5LKNdnGNg/sIOnl6SJEmSJEnToZcOgD8DOLuqNo10sSl2fQZ4V1X9cJy5rmz+Xgusa7q7HgfuBvZsrp2SZA3w9WZsH4Cq+gTt7ZAnMkpRrapWVlWrqloDOy7o9PkkSZIkSZI0RbPWmZVkb2ALcP8oIYcAxyf5ILAz8GSSx6rq3CTb0y5kXVJVn+1gucebv58c8nnr92clOQJ4LfCKqno0yXXAnCbPHYE9mvh5wCMdPaAkSZIkSZJm3KwUs5IsBM4Dzq2qGimmqg4fEn8GsKkpZAW4ALijqs6appQWAA81hazFwMuHXDsTuAT4DnA+cMw0rSlJkiRJkqQpmsli1twkq4Htab+R8GLabyqcqMOAtwFrm/kA/qiqvjiF3K4BTkxyB/BN2lsNSfIq4CDgsKrakuTXkryjqi4cbaL9d1/A4IqlU0hFkiRJkiRJncoojVLqUKvVqsHBwW6nIUmSJEmStM1IsqqqWiNd66UD4CVJkiRJkqQxzdoB8FslOYr2uVRD3VNVx01wno/S3oI41DljbQmUJEmSJElSf5v1YlZVXQtcOw3znDQN6UiSJEmSJKmPuM1QkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6huzfgD8tmbtho0sWn51t9OQRrR+xdJupyBJkiRJ0rSyM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOkbsq5vYrX8eS3LsGPHXJWlNIqdjk7x4ovdJkiRJkiRp5vTKNsPNVbUEIMlRwJ8DrxopsKq+CmyN/Tng28CXZyCnY4GrgNtnYG5JkiRJkiRNQk90Zg0zH3iow9jjgS9V1aOdBCf5eJLBJOuSfGDI+Ioktye5NcmHm86wNwAfarq/XjDhp5AkSZIkSdK065XOrLlJVgNzgN2AIzu87y3AWRNY531V9WCSAeArSQ4ANgDHAYurqpLsXFUPJ7kSuKqqLh8+SZJlwDKAgfkLJ7C8JEmSJEmSpqJXOrM2V9WSqloMHA18MknGuiHJbsD+wLUTWOdNSW4GbgH2BV4MbAQeAy5I8kZg3C6vqlpZVa2qag3suGACy0uSJEmSJGkqeqWY9ZSq+hqwCzBey9ObgCuq6iedzJtkL+A04DVVdQBwNTCnqp4ADgYuB44Brpls7pIkSZIkSZpZPVfMSrIYGAAeGCf0N4C/ncDU84EfARuTPBd4fbPePGBBVX0ReDdwYBP/CLDTBOaXJEmSJEnSDOu1M7MAAry9qraMFpxkEbAn8A+dLlBVa5LcAtwJ3Avc0FzaCfh8kjnN2u9pxj8FnJ/kFOD4qrprpHn3330BgyuWdpqGJEmSJEmSpqAnillVNTDB+PXA7h3GHjHk8wmjhB08wn030D5TS5IkSZIkST2i57YZSpIkSZIkSaPpic6skSTZH7h42PDjVXXIKPFXAHsNGz69qibytkNJkiRJkiT1sJ4tZlXVWmDJBOKPm7lsJEmSJEmS1AvcZihJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1jZ49AL5frN2wkUXLr+52GtrGrV+xtNspSJIkSZLUE+zMkiRJkiRJUt+YtWJWki1JVidZl2RNklOTjLt+kucl2ZTktGHjA0luSXLVOPdfl6Q1iXyPTfLiid4nSZIkSZKkmTObnVmbq2pJVe0LvA54PfD+Du47C/jSCOPvBO6YxvyGOxawmCVJkiRJktRDurLNsKruB5YBJyfJaHFJjgXuAdYNG98DWAp8YiLrJvl4ksGmO+wDQ8ZXJLk9ya1JPpzkUOANwIeabrIXTGQdSZIkSZIkzYyuHQBfVXcnGQB2Bb4//HqSecDptLu4Tht2+S+A/wbsNMFl31dVDzbrfiXJAcAG4DhgcVVVkp2r6uEkVwJXVdXlI+S2jHYxjoH5CyeYgiRJkiRJkiarlw+APwM4u6o2DR1Mcgxwf1WtmsScb0pyM3ALsC/tbYQbgceAC5K8EXh0vEmqamVVtaqqNbDjgkmkIUmSJEmSpMnoWmdWkr2BLcD9o4QcAhyf5IPAzsCTSR4DdgfekOSXgTnA/CR/U1VvHWe9vWh3eB1UVQ8luQiYU1VPJDkYeA1wPHAycOSUH1CSJEmSJEnTrivFrCQLgfOAc6uqRoqpqsOHxJ8BbKqqc5uh9zbjRwCnjVfIaswHfgRsTPJc2gfQX9dsZ9yxqr6Y5Abg7ib+ESa+jVGSJEmSJEkzaDaLWXOTrAa2B54ALqb9psJZUVVrktwC3AncC9zQXNoJ+HySOUCA9zTjnwLOT3IKcHxV3TXSvPvvvoDBFUtnNnlJkiRJkiQBkFEao9ShVqtVg4OD3U5DkiRJkiRpm5FkVVW1RrrWywfAS5IkSZIkST+lawfAb5XkKODMYcP3VNVxE5znCmCvYcOnV9W1U8lPkiRJkiRJvaPrxaym2DTlgtNEi1+SJEmSJEnqP24zlCRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvpG1w+A73drN2xk0fKru52GtnHrVyztdgqSJEmSJPUEO7MkSZIkSZLUN2asmJVkS5LVSdYlWZPk1CTjrpfkeUk2JTltyNj6JGub+QZnKmdJkiRJkiT1tpncZri5qpYAJNkVuBSYD7x/nPvOAr40wvirq+oH05qhJEmSJEmS+sqsbDOsqvuBZcDJSTJaXJJjgXuAdZNdK8l1Sc5OMpjkjiQHJflskm8l+dMhcZ9LsqrpHFvWjD2/idslyXZJrk/yS5PNRZIkSZIkSdNr1g6Ar6q7kwwAuwLfH349yTzgdOB1wGnDbwe+nKSA/1NVK8dZ7sdV1UryTuDzwMuAB4G7kpxdVQ8Av11VDyaZC9yU5DNV9Z0kZwIfB74B3F5VXx4h12W0i3MMzF/Y8X8DSZIkSZIkTU0vvc3wDODsqto0QvPWL1bVhma74t8nubOq/nGMua5s/l4LrKuq7wIkuRvYE3gAOCXJcU3cnsA+wANV9Ykkvw6cCCwZafKmmLYSYIfd9qmJPaYkSZIkSZIma9aKWUn2BrYA948ScghwfJIPAjsDTyZ5rKrOraoN0N6umOQK4GBgrGLW483fTw75vPX7s5IcAbwWeEVVPZrkOmBOk+eOwB5N/DzgkQk8piRJkiRJkmbQrBSzkiwEzgPOraoRO5mq6vAh8WcAm6rq3CQ/A2xXVY80n38J+JMpprQAeKgpZC0GXj7k2pnAJcB3gPOBY6a4liRJkiRJkqbJTBaz5iZZDWwPPAFcTPtNhRP1XOCKZuvhs4BLq+qaKeZ2DXBikjuAbwJfB0jyKuAg4LCq2pLk15K8o6ounOJ6kiRJkiRJmgYZpVFKHWq1WjU4ONjtNCRJkiRJkrYZSVZVVWuka9vNdjKSJEmSJEnSZM362wyTHEX7XKqh7qmq40aKH2OejwKHDRs+xy2BkiRJkiRJ265ZL2ZV1bXAtdMwz0nTkI4kSZIkSZL6iNsMJUmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9Y9bfZritWbthI4uWX93tNLSNW79iabdTkCRJkiSpJ8x6Z1aS9yVZl+TWJKuTHDJK3CVJvpnktiR/lWT7ZvxXh9w7mOQXx1hrUZLbJpnnH03mPkmSJEmSJM2cWS1mJXkFcAzw0qo6AHgtcO8o4ZcAi4H9gbnA7zbjXwEOrKolwG8Dn5ihdC1mSZIkSZIk9ZjZ3ma4G/CDqnocoKp+MFpgVX1x6+ck3wD2aMY3DQn7GaA6WTjJIuDi5h6Ak6vqn5PsBlwGzKf93+O/AkuBuUlWA+uq6jc7WUOSJEmSJEkza7a3GX4Z2DPJvyT5WJJXjXdDs73wbcA1Q8aOS3IncDXt7qxO3A+8rqpeCrwZ+Mtm/D8D1zadXgcCq6tqObC5qpaMVMhKsqzZ4ji45dGNHS4vSZIkSZKkqZrVYlbTVfUyYBnw78BlSU4Y57aPAf9YVdcPmeeKqloMHAv8zw6X3x44P8la4NPAi5vxm4B3JDkD2L+qHungOVZWVauqWgM7LuhweUmSJEmSJE3VrB8AX1Vbquq6qno/cDLwa6PFJnk/sBB4zyhz/SOwd5JdOlj63cD3aXdftYBnD5njlcAG4KIkvzWBx5EkSZIkSdIsmu0D4F+UZJ8hQ0uA74wS+7vAUcBvVNWTQ8ZfmCTN55cCOwAPdLD8AuC7zVxvAwaaOZ4PfL+qzqd9mPxLm/ifbH2DoiRJkiRJknrDbB8APw/4SJKdgSeAb9PecjiS82gXur7W1K4+W1V/QruT67eS/ATYDLy5qjo5BP5jwGeazqtrgB8140cAf9jMtwnY2pm1Erg1yc0eAC9JkiRJktQb0lkdSKNptVo1ODjY7TQkSZIkSZK2GUlWVVVrpGuzfmaWJEmSJEmSNFmzvc3waZJcAew1bPj0qrp2AnPsD1w8bPjxqjpkqvlJkiRJkiSpd3S9mFVVx03DHGtpHyYvSZIkSZKkbZjbDCVJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfaPrbzPsd2s3bGTR8qu7nYb6xPoVS7udgiRJkiRJfa0rnVlJtiRZnWRNkpuTHDpG7PObmNVJ1iU5cTZzlSRJkiRJUu/oVmfW5qpaApDkKODPgVeNEvtd4BVV9XiSecBtSa6sqn+bnVQlSZIkSZLUK3rhzKz5wEOjXayqH1fV483XHRgn5ySbknyo6eL6v0kOTnJdkruTvKGJWZTk+qbj66nOsCTHJflK2nZL8i9J/r9pek5JkiRJkiRNUbc6s+YmWQ3MAXYDjhwrOMmewNXAC4E/HKcr62eA/1dVf5jkCuBPgdcBLwb+GrgSuB94XVU9lmQf4G+BVlVdkeTXgJOAo4H3V9X3RshnGbAMYGD+ws6fWpIkSZIkSVPSC9sMXwF8Msl+VVUjBVfVvcABSX4e+FySy6vq+6PM/WPgmubzWuDxqvpJkrXAomZ8e+DcJEuALcB/GnL/HwC3AV+vqr8dJZ+VwEqAHXbbZ8ScJUmSJEmSNP26vs2wqr4G7AKM2+LUdGTdBhw+RthPhhTFngQeb+59kv8o3r0b+D5wINACnj3k/j2a+56bpOv/fSRJkiRJkvQful6sSbIYGAAeGOX6HknmNp9/FvhF4JtTXHYB8N2mwPW2Zn2SPAv4K+A3gDuA90xxHUmSJEmSJE2jbp+ZBRDg7VW1ZZTYXwD+d5JqYj9cVWunuP7HgM8k+S3aWxJ/1Iz/EXB9Vf1TkjXATUmurqo7prieJEmSJEmSpkFGOaZKHWq1WjU4ONjtNCRJkiRJkrYZSVZVVWuka13fZihJkiRJkiR1qlvbDJ8myf7AxcOGH6+qQ0aJvxHYYdjw26ZhC6IkSZIkSZJ6VM8Us5oi1JIJxI9Y5JIkSZIkSdK2y22GkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+0TNvM+xXazdsZNHyq7udhvrE+hVLu52CJEmSJEl9rSc6s5JsSbI6yW1JvpBk5zFin5/k5iZ+XZITZzFVSZIkSZIkdVFPFLOAzVW1pKr2Ax4EThoj9rvAK6pqCXAIsDzJz89CjpIkSZIkSeqyXilmDfU1YPfRLlbVj6vq8ebrDozzDEk+nmSw6eL6QDN2dJJPD4k5IslVzeffSfIvSb6R5Pwk5075iSRJkiRJkjQteqqYlWQAeA1w5Thxeya5FbgXOLOq/m2M8PdVVQs4AHhVkgOA/wsckuRnmpg3A59qOrz+B/By4DBg8SjrL2sKZINbHt04gSeUJEmSJEnSVPRKMWtuktXA94DnAn8/VnBV3VtVBwAvBN6e5LljhL8pyc3ALcC+wIur6gngGuBXkjwLWAp8HjgY+IeqerCqfgJ8eqQJq2plVbWqqjWw44IJPagkSZIkSZImr1eKWZubM7CeD4Sxz8x6StORdRtw+EjXk+wFnAa8pil+XQ3MaS5/CngTcCQwWFWPTOUBJEmSJEmSNPN6pZgFQFU9CpwCnNp0TD1Nkj2SzG0+/yzwi8A3R5lyPvAjYGPTvfX6Idf+AXgp8Hu0C1sAN9Heivizzfq/NsVHkiRJkiRJ0jTqqWIWQFXdAtwK/MYoIb8A3JhkDe2C1Ierau0oc62hvb3wTuBS4IYh17YAV9EucF3VjG0A/hfwjSZ2PeChWJIkSZIkST0iVdXtHHpKknlVtanpzLoC+KuqumK0+FarVYODg7OXoCRJkiRJ0jYuyarmhX5P03OdWT3gjOYw+tuAe4DPdTUbSZIkSZIkPWXEc6l6QZL9gYuHDT9eVYeMEn8jsMOw4beNtgVxNFV12kTiJUmSJEmSNHt6tpjVFKGWTCB+xCKXJEmSJEmSth1uM5QkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Y2efZthv1i7YSOLll/d7TQ0y9avWNrtFCRJkiRJekaalc6sJFuSrE6yLsmaJKcmGXftJM9LsinJaUPGdk5yeZI7k9yR5BUzm70kSZIkSZJ6xWx1Zm2uqiUASXYFLgXmA+8f576zgC8NGzsHuKaqjk/ybGDHac5VkiRJkiRJPWrWz8yqqvuBZcDJSTJaXJJjgXuAdUPGFgCvBC5o5vpxVT08xhy/l+SmphvsM0l2TLIgyXe2doYl+Zkk9ybZPslBSW5tusg+lOS26XhmSZIkSZIkTY+uHABfVXcDA8CuI11PMg84HfjAsEt7Af8OXJjkliSfSPIzYyz12ao6qKoOBO4AfqeqNgKrgVc1MccA11bVT4ALgf/SdJFtmdTDSZIkSZIkacb06tsMzwDOrqpNw8afBbwU+HhVvQT4EbB8jHn2S3J9krXAbwL7NuOXAW9uPr8FuCzJzsBOVfW1ZvzS0SZNsizJYJLBLY9unMBjSZIkSZIkaSq68jbDJHvT7ny6f5SQQ4Djk3wQ2Bl4MsljwOXAfVV1YxN3OWMXsy4Cjq2qNUlOAI5oxq8E/leSnwNeBvw/YKdO86+qlcBKgB1226c6vU+SJEmSJElTM+vFrCQLgfOAc6tqxEJQVR0+JP4MYFNVndt8vzfJi6rqm8BrgNvHWG4n4LtJtqfdmbWhmX9TkptoHyZ/VVVtAR5O8kiSQ5pi2Vum+qySJEmSJEmaXrNVzJqbZDWwPfAEcDHtNxVOxh8AlzRvMrwbeMcYsf8DuJH2OVs38tPdV5cBn+Y/urUAfgc4P8mTwD8A7iGUJEmSJEnqIRmlOeoZKcm8red0JVkO7FZV7xzrnlarVYODg7OSnyRJkiRJ0jNBklVV1RrpWlfOzOphS5O8l/Z/l+8AJ3Q3HUmSJEmSJA3V1WJWkqOAM4cN31NVx01wno8Chw0bPqeqLpzIPFV1Ge3th5IkSZIkSepBXS1mVdW1wLXTMM9J05COJEmSJEmSetx23U5AkiRJkiRJ6pTFLEmSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvdPVthtuCtRs2smj51d1OQ7Ns/Yql3U5BkiRJkqRnJDuzJEmSJEmS1Dd6ppiVZEuS1UluS/LpJDuOEjcnyTeSrEmyLskHxpn3uiStSeRzbJIXT/Q+SZIkSZIkzZyeKWYBm6tqSVXtB/wYOHGUuMeBI6vqQGAJcHSSl89APscCFrMkSZIkSZJ6SC8Vs4a6HnjhSBeqbVPzdfvmT3UyaZKPJxkc3tGVZEWS25PcmuTDSQ4F3gB8qOkWe8HUHkeSJEmSJEnToecOgE/yLOD1wDVjxAwAq2gXvD5aVTd2OP37qurB5v6vJDkA2AAcByyuqkqyc1U9nORK4KqqunyE9ZcBywAG5i+cyONJkiRJkiRpCnqpM2tuktXAIPCvwAWjBVbVlqpaAuwBHJxkvw7XeFOSm4FbgH1pbyPcCDwGXJDkjcCj401SVSurqlVVrYEdF3S4tCRJkiRJkqaqlzqzNjcFqo41HVRfBY4GbhsrNslewGnAQVX1UJKLgDlV9USSg4HXAMcDJwNHTiJ/SZIkSZIkzbBe6szqSJKFSXZuPs8FXgfc2cGt84EfARuTPJf2VkaSzAMWVNUXgXcDBzbxjwA7TW/2kiRJkiRJmope6szq1G7AXzfnXm0H/F1VXTXeTVW1JskttAtf9wI3NJd2Aj6fZA4Q4D3N+KeA85OcAhxfVXeNNO/+uy9gcMXSKT2QJEmSJEnSWH7yk59w33338dhjj3U7lWk1Z84c9thjD7bffvuO70lVRy8C1CharVYNDg52Ow1JkiRJkrQNu+eee9hpp514znOeQ5JupzMtqooHHniARx55hL322uunriVZVVWtke7ru22GkiRJkiRJzzSPPfbYNlXIAkjCc57znAl3m/XsNsMkzwG+MsKl11TVAyPEXwHsNWz49Kq6dibykyRJkiRJmk3bUiFrq8k8U88Ws5qC1ZIJxB83c9lIkiRJkiQ9sx166KH88z//c7fT6N1iliRJkiRJkka2aPnV0zrf+g5ebtcLhSzwzCxJkiRJkiR1YN68eQBcd911vOpVr+JXf/VX2XvvvVm+fDmXXHIJBx98MPvvvz933XUXACeccAInnngirVaL//Sf/hNXXXXVtORhZ5YkSZIkSZImZM2aNdxxxx383M/9HHvvvTe/+7u/yze+8Q3OOeccPvKRj/AXf/EXAKxfv55vfOMb3HXXXbz61a/m29/+NnPmzJnS2nZmSZIkSZIkaUIOOuggdtttN3bYYQde8IIX8Eu/9EsA7L///qxfv/6puDe96U1st9127LPPPuy9997ceeedU17bzqwpWrth47TvU9Xs6WRPsCRJkiRJ+mk77LDDU5+32267p75vt912PPHEE09dG/62wul4I6OdWZIkSZIkSZoRn/70p3nyySe56667uPvuu3nRi1405TlnrDMryRZgLbA98ATwSeDsqnpylPhFwB3AN5uhr1fVic21lwEXAXOBLwLvrKqaqdwlSZIkSZI0dc973vM4+OCD+eEPf8h555035fOyYGa3GW6uqiUASXYFLgXmA+8f4567tt4zzMeB3wNupF3MOhr40nQmK0mSJEmS1C+6cWzOpk2bADjiiCM44ogjnhq/7rrrnvo8/NprX/tazjvvvGnNY1a2GVbV/cAy4ORMcHNkkt2A+VX19aYb65PAsWPEX5fk7CSDSe5IclCSzyb5VpI/HRL3uSSrkqxLsqwZe34Tt0uS7ZJcn+SXJvPMkiRJkiRJmn6zdgB8Vd2dZADYFfj+KGF7JbkF+CHw36vqemB34L4hMfc1Y2P5cVW1krwT+DzwMuBB4K4kZ1fVA8BvV9WDSeYCNyX5TFV9J8mZtDvBvgHcXlVfHj55U/xaBjAwf2Fn/wEkSZIkSZKeQS666KIZmbeX3mb4XeB5VfVAc0bW55LsO8m5rmz+Xgusq6rvAiS5G9gTeAA4JclxTdyewD7AA1X1iSS/DpwILBlp8qpaCawE2GG3fTy7S5IkSZIkaZbMWjEryd7AFuD+ka5X1ePA483nVUnuAv4TsAHYY0joHs3YWB5v/n5yyOet35+V5AjgtcArqurRJNcBc5o8dxyy3jzgkfGfTpIkSZIkaWZVFRM8vannTeb9frNyZlaShcB5wLmjvYUwycJmG+LWwtc+wN1NV9UPk7y8OW/rt2hvHZyKBcBDTSFrMfDyIdfOBC4B/hg4f4rrSJIkSZIkTdmcOXN44IEHJlX86VVVxQMPPDDhNxzOZGfW3CSrge2BJ4CLgbPGiH8l8CdJfkK7g+rEqnqwufb7wEXAXNpvMZzqmwyvAU5McgfwTeDrAEleBRwEHFZVW5L8WpJ3VNWFo020/+4LGOzCGwQkSZIkSdIzxx577MF9993Hv//7v3c7lWk1Z84c9thjj/EDh8i2VNHrhlarVYODg91OQ5IkSZIkaZuRZFVVtUa6NivbDCVJkiRJkqTpMOtvM0xyFO1zqYa6p6qOGyl+jHk+Chw2bPicsbYESpIkSZIkqb/NejGrqq4Frp2GeU6ahnQkSZIkSZLURzwza4qSPEL7EHmpE7sAP+h2Euob/l40Ef5eNBH+XtQpfyuaCH8vmgh/LxrP86tq4UgXZr0zaxv0zdEOJJOGSzLo70Wd8veiifD3oonw96JO+VvRRPh70UT4e9FUeAC8JEmSJEmS+obFLEmSJEmSJPUNi1lTt7LbCaiv+HvRRPh70UT4e9FE+HtRp/ytaCL8vWgi/L1o0jwAXpIkSZIkSX3DzixJkiRJkiT1DYtZU5Dk6CTfTPLtJMu7nY96S5K/SnJ/ktuGjP1ckr9P8q3m75/tZo7qDUn2TPLVJLcnWZfknc24vxc9TZI5Sb6RZE3ze/lAM75Xkhubf5MuS/Lsbueq3pFkIMktSa5qvvt70YiSrE+yNsnqJIPNmP8eaURJdk5yeZI7k9yR5BX+XjSSJC9q/ndl658fJnmXvxdNlsWsSUoyAHwUeD3wYuA3kry4u1mpx1wEHD1sbDnwlaraB/hK8116Aji1ql4MvBw4qfnfE38vGsnjwJFVdSCwBDg6ycuBM4Gzq+qFwEPA73QvRfWgdwJ3DPnu70VjeXVVLamqVvPdf480mnOAa6pqMXAg7f+d8feip6mqbzb/u7IEeBnwKHAF/l40SRazJu9g4NtVdXdV/Rj4FPCrXc5JPaSq/hF4cNjwrwJ/3Xz+a+DY2cxJvamqvltVNzefH6H9fwjujr8XjaDaNjVft2/+FHAkcHkz7u9FT0myB7AU+ETzPfh70cT475GeJskC4JXABQBV9eOqehh/Lxrfa4C7quo7+HvRJFnMmrzdgXuHfL+vGZPG8tyq+m7z+XvAc7uZjHpPkkXAS4Ab8feiUTRbxlYD9wN/D9wFPFxVTzQh/pukof4C+G/Ak8335+DvRaMr4MtJViVZ1oz575FGshfw78CFzTbmTyT5Gfy9aHxvAf62+ezvRZNiMUvqkmq/StTXieopSeYBnwHeVVU/HHrN34uGqqotTZv+HrQ7hRd3NyP1qiTHAPdX1apu56K+8YtV9VLaR2mclOSVQy/675GGeBbwUuDjVfUS4EcM2yLm70XDNWc0vgH49PBr/l40ERazJm8DsOeQ73s0Y9JYvp9kN4Dm7/u7nI96RJLtaReyLqmqzzbD/l40pmY7x1eBVwA7J3lWc8l/k7TVYcAbkqynfSTCkbTPuPH3ohFV1Ybm7/tpn2dzMP57pJHdB9xXVTc23y+nXdzy96KxvB64uaq+33z396JJsZg1eTcB+zRvA3o27VbJK7uck3rflcDbm89vBz7fxVzUI5rzay4A7qiqs4Zc8veip0myMMnOzee5wOton7P2VeD4JszfiwCoqvdW1R5VtYj2/63y/6rqN/H3ohEk+ZkkO239DPwScBv+e6QRVNX3gHuTvKgZeg1wO/5eNLbf4D+2GIK/F01S2p18mowkv0z7HIoB4K+q6s+6m5F6SZK/BY4AdgG+D7wf+Bzwd8DzgO8Ab6qq4YfE6xkmyS8C1wNr+Y8zbf6I9rlZ/l70U5IcQPuA1AHa/0+pv6uqP0myN+3Om58DbgHeWlWPdy9T9ZokRwCnVdUx/l40kuZ3cUXz9VnApVX1Z0meg/8eaQRJltB+ucSzgbuBd9D824S/Fw3TFMn/Fdi7qjY2Y/7viybFYpYkSZIkSZL6htsMJUmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqG/8/Bkfq0YINPYMAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# feature importance for top 30 features\n", + "fea_imp = pd.DataFrame({'imp':model.feature_importances_, 'col': features})\n", + "fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n", + "_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Evaluation" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "tags": [ + "block:evaluation_result", + "prev:modelling" + ] + }, + "outputs": [], + "source": [ + "model = joblib.load('lgb.jl')" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "binary_logloss = model.booster_.best_score.get('valid_0').get('binary_logloss')" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "amex_metric_score = model.booster_.best_score.get('valid_0').get('amex_metric_score')" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.24605493989573005\n" + ] + } + ], + "source": [ + "print(binary_logloss)" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "tags": [ + "pipeline-metrics" + ] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.7625788091718922\n" + ] + } + ], + "source": [ + "print(amex_metric_score)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "tags": [] + }, + "source": [ + "## Submission" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "sub = pd.DataFrame({'customer_ID': test.index,\n", + " 'prediction': np.mean(y_pred_list, axis=0)})\n", + "sub.to_csv('submission.csv', index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "skip" + ] + }, + "outputs": [], + "source": [ + "sub" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "kubeflow_notebook": { + "autosnapshot": true, + "experiment": { + "id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc", + "name": "g-research-crypto-forecasting" + }, + "experiment_name": "g-research-crypto-forecasting", + "katib_metadata": { + "algorithm": { + "algorithmName": "grid" + }, + "maxFailedTrialCount": 3, + "maxTrialCount": 12, + "objective": { + "objectiveMetricName": "", + "type": "minimize" + }, + "parallelTrialCount": 3, + "parameters": [] + }, + "katib_run": false, + "pipeline_description": "forecasting short term returns in 14 popular cryptocurrencies.", + "pipeline_name": "g-research-crypto-forecasting-pipeline", + "snapshot_volumes": true, + "steps_defaults": [ + "label:access-ml-pipeline:true", + "label:kaggle-secret:true", + "label:access-rok:true" + ], + "volume_access_mode": "rwm", + "volumes": [ + { + "annotations": [], + "mount_point": "/home/jovyan", + "name": "test-workspace-qtvmt", + "size": 32, + "size_type": "Gi", + "snapshot": false, + "type": "clone" + } + ] + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.9" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/american-express-default-kaggle-competition/requirements.txt b/american-express-default-kaggle-competition/requirements.txt index 22259d0c..751ba674 100644 --- a/american-express-default-kaggle-competition/requirements.txt +++ b/american-express-default-kaggle-competition/requirements.txt @@ -1,7 +1,7 @@ -kaggle -pandas -tqdm -wget -lightgbm -pyarrow -fastparquet +kaggle +pandas +tqdm +wget +lightgbm +pyarrow +fastparquet diff --git a/bluebook-for-bulldozers-kaggle-competition/Readme.md b/bluebook-for-bulldozers-kaggle-competition/Readme.md index 531a8a76..71e58a47 100644 --- a/bluebook-for-bulldozers-kaggle-competition/Readme.md +++ b/bluebook-for-bulldozers-kaggle-competition/Readme.md @@ -1,348 +1,348 @@ -# Objective - -This example is based on the Bluebook for bulldozers competition (https://www.kaggle.com/competitions/bluebook-for-bulldozers/overview). The objective of this exercise is to predict the sale price of bulldozers sold at auctions. - -## Environment - -This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 and ARM based system which includes all Intel and AMD based CPU's and M1/M2 series Macbooks. - -## Step 1: Setup Kubeflow as a Service - -- If you haven’t already, sign up (https://www.arrikto.com/kubeflow-as-a-service/) -- Deploy Kubeflow - -## Step 2: Launch a Notebook Server - -- Bump memory to 2GB and vCPUs to 2 - - -## Step 3: Clone the Project Repo to Your Notebook - -- (Kubeflow as a Service) Open up a terminal in the Notebook Server and git clone the kubeflow/examples repository -``` -git clone https://github.com/kubeflow/examples -``` - -## Step 4: Setup DockerHub and Docker - -- If you haven’t already, sign up (https://hub.docker.com/) for DockerHub -- If you haven’t already, install Docker Desktop (https://www.docker.com/products/docker-desktop/) locally OR install the Docker command line utility (https://docs.docker.com/get-docker/) and enter `sudo docker login` command in your terminal and log into Docker with your your DockerHub username and password - - -## Step 5: Setup Kaggle - -- If you haven’t already done so, sign up (https://www.kaggle.com/) for Kaggle -- (On Kaggle) Generate an API token (https://www.kaggle.com/docs/api) -- (Kubeflow as a Service) Create a Kubernetes secret -``` -kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= -``` - -## Step 6: Install Git - -- (Locally) If you don’t have it already, install Git (https://github.com/git-guides/install-git) - -## Step 7: Clone the Project Repo Locally - -- (Locally) Git clone the `kubeflow/examples` repository -``` -git clone https://github.com/kubeflow/examples -``` - -## Step 8: Create a PodDefault Resource - -- (Kubeflow as a Service) Navigate to the `bluebook-for-bulldozers-kaggle-competition directory` -- Create a resource.yaml file - -resource.yaml: -``` -apiVersion: "kubeflow.org/v1alpha1" -kind: PodDefault -metadata: - name: kaggle-access -spec: - selector: - matchLabels: - kaggle-secret: "true" - desc: "kaggle-access" - volumeMounts: - - name: secret-volume - mountPath: /secret/kaggle - volumes: - - name: secret-volume - secret: - secretName: kaggle-secret -``` - -image3 - -- Apply resource.yaml using `kubectl apply -f resource.yaml` - -## Step 9: Explore the load-data directory - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/load-data` directory -- Open up the `load.py` file -- Note the code in this file that will perform the actions required in the “load-data” pipeline step - -image7 - -## Step 10: Build the load Docker Image - -- (Locally) Navigate to the bluebook-for-bulldozers-kaggle-competition/pipeline-components/load-data directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 11: Push the load Docker Image to DockerHub - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/load-data` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 12: Explore the preprocess directory - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/preprocess` directory -- Open up the `preprocess.py` file -- Note the code in this file that will perform the actions required in the “preprocess” pipeline step - -image5 - -## Step 13: Build the preprocess Docker Image - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/preprocess` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` -## Step 14: Push the preprocess Docker Image to DockerHub - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/preprocess` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 15: Explore the train directory - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/train` directory -- Open up the train.py file -- Note the code in this file that will perform the actions required in the “train” pipeline step - - -![image2](https://user-images.githubusercontent.com/17012391/177051233-a32e87db-7771-4b5f-9afe-141063733262.png) - -## Step 16: Build the train Docker Image - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/train` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 17: Push the train Docker Image to DockerHub - -- (Locally) Navigate to the bluebook-for-bulldozers-kaggle-competition/pipeline-components/train directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 18: Explore the test directory - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/test` directory -- Open up the `test.py` file -- Note the code in this file that will perform the actions required in the “test” pipeline step - -image6 - -## Step 19: Build the test Docker Image - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/test` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` -## Step 20: Push the test Docker Image to DockerHub - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/test` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 21: Modify the blue-book-for-bulldozers-kfp.py file - -(Kubeflow as a Service) Navigate to the `bluebook-for-bulldozers-kaggle-competition` directory -Update the `bluebook-for-bulldozers-kaggle-competition-kfp.py` with accurate Docker Image inputs - -``` - return dsl.ContainerOp( - name = 'load-data', - image = '/:', - -—----- - -def PreProcess(comp1): - return dsl.ContainerOp( - name = 'preprocess', - image = '/:', - -—----- - -def Train(comp2): - return dsl.ContainerOp( - name = 'train', - image = '/:', - -—----- - -def Test(comp3): - return dsl.ContainerOp( - name = 'test', - image = '/:', - - ``` - -## Step 22: Generate a KFP Pipeline yaml File - -- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition` directory and delete the existing `blue-book-for-bulldozers-kaggle-competition-kfp.yaml` file -- (Kubeflow as a Service) Navigate to the `bluebook-for-bulldozers-kaggle-competition` directory - -Build a python virtual environment: - -Step a) Update pip -``` -python3 -m pip install --upgrade pip -``` - -Step b) Install virtualenv -``` -sudo pip3 install virtualenv -``` - -Step c) Check the installed version of venv -``` -virtualenv --version -``` - -Step d) Name your virtual enviornment as kfp -``` -virtualenv kfp -``` - -Step e) Activate your venv. -``` -source kfp/bin/activate -``` - -After this virtual environment will get activated. Now in our activated venv we need to install following packages: -``` -sudo apt-get update -sudo apt-get upgrade -sudo apt-get install -y git python3-pip - -python3 -m pip install kfp==1.1.2 -``` - -After installing packages create the yaml file - -Inside venv point your terminal to a path which contains our kfp file to build pipeline (blue-book-for-bulldozers-kaggle-competition-kfp.py) and run these commands to generate a `yaml` file for the Pipeline: - -``` -blue-book-for-bulldozers-kaggle-competition-kfp.py -``` - -Screenshot 2022-07-04 at 12 01 51 AM - -- Download the `bluebook-for-bulldozers-kaggle-competition.yaml` file that was created to your local `bluebook-for-bulldozers-kaggle-competition` directory - -## Step 23: Create an Experiment - -- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Experiments (KFP) > Create Experiment view -- Name the experiment and click Next -- Click on Experiments (KFP) to view the experiment you just created - -## Step 24: Create a Pipeline - -- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Pipelines > +Upload Pipeline view -- Name the pipeline -- Click on Upload a file -- Upload the local bluebook-for-bulldozers-kaggle-competition.py.yaml file -- Click Create - -Step 25: Create a Run - -- (Kubeflow as a Service) Click on Create Run in the view from the previous step -- Choose the experiment we created in Step 23 -- Click Start -- Click on the run name to view the runtime execution graph - -Screenshot 2022-07-04 at 12 04 43 AM - - -## Troubleshooting Tips: -While running the pipeline as mentioned above you may come across this error: -![kaggle-secret-error-01](https://user-images.githubusercontent.com/17012391/175290593-aac58d80-0d9f-47bd-bd20-46e6f5207210.PNG) - -errorlog: - -``` -kaggle.rest.ApiException: (403) -Reason: Forbidden -HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Thu, 23 Jun 2022 11:31:18 GMT', 'Access-Control-Allow-Credentials': 'true', 'Set-Cookie': 'ka_sessionid=6817a347c75399a531148e19cad0aaeb; max-age=2626560; path=/, GCLB=CIGths3--ebbUg; path=/; HttpOnly', 'Transfer-Encoding': 'chunked', 'Vary': -HTTP response body: b'{"code":403,"message":"You must accept this competition\\u0027s rules before you\\u0027ll be able to download files."}' - -``` -This error occours for two reasons: -- Your Kaggle account is not verified with your phone number. -- Rules for this specific competitions are not accepted. - -Lets accept Rules of Bulldozers competition -![kaggle-secret-error-02](https://user-images.githubusercontent.com/17012391/175291406-7a30e06d-fc05-44c3-b33c-bccd31b381bd.PNG) - -Click on "I Understand and Accept". After this you will be prompted to verify your account using your phone number: -![kaggle-secret-error-03](https://user-images.githubusercontent.com/17012391/175291608-daad1a47-119a-4e47-b48b-4f878d65ddd7.PNG) - -Add your phone number and Kaggle will send the code to your number, enter this code and verify your account. ( Note: pipeline wont run if your Kaggle account is not verified ) - -## Success -After the kaggle account is verified pipeline run is successful we will get the following: - -Screenshot 2022-06-10 at 12 04 48 AM - - +# Objective + +This example is based on the Bluebook for bulldozers competition (https://www.kaggle.com/competitions/bluebook-for-bulldozers/overview). The objective of this exercise is to predict the sale price of bulldozers sold at auctions. + +## Environment + +This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 and ARM based system which includes all Intel and AMD based CPU's and M1/M2 series Macbooks. + +## Step 1: Setup Kubeflow as a Service + +- If you haven’t already, sign up (https://www.arrikto.com/kubeflow-as-a-service/) +- Deploy Kubeflow + +## Step 2: Launch a Notebook Server + +- Bump memory to 2GB and vCPUs to 2 + + +## Step 3: Clone the Project Repo to Your Notebook + +- (Kubeflow as a Service) Open up a terminal in the Notebook Server and git clone the kubeflow/examples repository +``` +git clone https://github.com/kubeflow/examples +``` + +## Step 4: Setup DockerHub and Docker + +- If you haven’t already, sign up (https://hub.docker.com/) for DockerHub +- If you haven’t already, install Docker Desktop (https://www.docker.com/products/docker-desktop/) locally OR install the Docker command line utility (https://docs.docker.com/get-docker/) and enter `sudo docker login` command in your terminal and log into Docker with your your DockerHub username and password + + +## Step 5: Setup Kaggle + +- If you haven’t already done so, sign up (https://www.kaggle.com/) for Kaggle +- (On Kaggle) Generate an API token (https://www.kaggle.com/docs/api) +- (Kubeflow as a Service) Create a Kubernetes secret +``` +kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= +``` + +## Step 6: Install Git + +- (Locally) If you don’t have it already, install Git (https://github.com/git-guides/install-git) + +## Step 7: Clone the Project Repo Locally + +- (Locally) Git clone the `kubeflow/examples` repository +``` +git clone https://github.com/kubeflow/examples +``` + +## Step 8: Create a PodDefault Resource + +- (Kubeflow as a Service) Navigate to the `bluebook-for-bulldozers-kaggle-competition directory` +- Create a resource.yaml file + +resource.yaml: +``` +apiVersion: "kubeflow.org/v1alpha1" +kind: PodDefault +metadata: + name: kaggle-access +spec: + selector: + matchLabels: + kaggle-secret: "true" + desc: "kaggle-access" + volumeMounts: + - name: secret-volume + mountPath: /secret/kaggle + volumes: + - name: secret-volume + secret: + secretName: kaggle-secret +``` + +image3 + +- Apply resource.yaml using `kubectl apply -f resource.yaml` + +## Step 9: Explore the load-data directory + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/load-data` directory +- Open up the `load.py` file +- Note the code in this file that will perform the actions required in the “load-data” pipeline step + +image7 + +## Step 10: Build the load Docker Image + +- (Locally) Navigate to the bluebook-for-bulldozers-kaggle-competition/pipeline-components/load-data directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 11: Push the load Docker Image to DockerHub + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/load-data` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 12: Explore the preprocess directory + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/preprocess` directory +- Open up the `preprocess.py` file +- Note the code in this file that will perform the actions required in the “preprocess” pipeline step + +image5 + +## Step 13: Build the preprocess Docker Image + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/preprocess` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` +## Step 14: Push the preprocess Docker Image to DockerHub + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/preprocess` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 15: Explore the train directory + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/train` directory +- Open up the train.py file +- Note the code in this file that will perform the actions required in the “train” pipeline step + + +![image2](https://user-images.githubusercontent.com/17012391/177051233-a32e87db-7771-4b5f-9afe-141063733262.png) + +## Step 16: Build the train Docker Image + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/train` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 17: Push the train Docker Image to DockerHub + +- (Locally) Navigate to the bluebook-for-bulldozers-kaggle-competition/pipeline-components/train directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 18: Explore the test directory + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/test` directory +- Open up the `test.py` file +- Note the code in this file that will perform the actions required in the “test” pipeline step + +image6 + +## Step 19: Build the test Docker Image + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/test` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` +## Step 20: Push the test Docker Image to DockerHub + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition/pipeline-components/test` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 21: Modify the blue-book-for-bulldozers-kfp.py file + +(Kubeflow as a Service) Navigate to the `bluebook-for-bulldozers-kaggle-competition` directory +Update the `bluebook-for-bulldozers-kaggle-competition-kfp.py` with accurate Docker Image inputs + +``` + return dsl.ContainerOp( + name = 'load-data', + image = '/:', + +—----- + +def PreProcess(comp1): + return dsl.ContainerOp( + name = 'preprocess', + image = '/:', + +—----- + +def Train(comp2): + return dsl.ContainerOp( + name = 'train', + image = '/:', + +—----- + +def Test(comp3): + return dsl.ContainerOp( + name = 'test', + image = '/:', + + ``` + +## Step 22: Generate a KFP Pipeline yaml File + +- (Locally) Navigate to the `bluebook-for-bulldozers-kaggle-competition` directory and delete the existing `blue-book-for-bulldozers-kaggle-competition-kfp.yaml` file +- (Kubeflow as a Service) Navigate to the `bluebook-for-bulldozers-kaggle-competition` directory + +Build a python virtual environment: + +Step a) Update pip +``` +python3 -m pip install --upgrade pip +``` + +Step b) Install virtualenv +``` +sudo pip3 install virtualenv +``` + +Step c) Check the installed version of venv +``` +virtualenv --version +``` + +Step d) Name your virtual enviornment as kfp +``` +virtualenv kfp +``` + +Step e) Activate your venv. +``` +source kfp/bin/activate +``` + +After this virtual environment will get activated. Now in our activated venv we need to install following packages: +``` +sudo apt-get update +sudo apt-get upgrade +sudo apt-get install -y git python3-pip + +python3 -m pip install kfp==1.1.2 +``` + +After installing packages create the yaml file + +Inside venv point your terminal to a path which contains our kfp file to build pipeline (blue-book-for-bulldozers-kaggle-competition-kfp.py) and run these commands to generate a `yaml` file for the Pipeline: + +``` +blue-book-for-bulldozers-kaggle-competition-kfp.py +``` + +Screenshot 2022-07-04 at 12 01 51 AM + +- Download the `bluebook-for-bulldozers-kaggle-competition.yaml` file that was created to your local `bluebook-for-bulldozers-kaggle-competition` directory + +## Step 23: Create an Experiment + +- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Experiments (KFP) > Create Experiment view +- Name the experiment and click Next +- Click on Experiments (KFP) to view the experiment you just created + +## Step 24: Create a Pipeline + +- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Pipelines > +Upload Pipeline view +- Name the pipeline +- Click on Upload a file +- Upload the local bluebook-for-bulldozers-kaggle-competition.py.yaml file +- Click Create + +Step 25: Create a Run + +- (Kubeflow as a Service) Click on Create Run in the view from the previous step +- Choose the experiment we created in Step 23 +- Click Start +- Click on the run name to view the runtime execution graph + +Screenshot 2022-07-04 at 12 04 43 AM + + +## Troubleshooting Tips: +While running the pipeline as mentioned above you may come across this error: +![kaggle-secret-error-01](https://user-images.githubusercontent.com/17012391/175290593-aac58d80-0d9f-47bd-bd20-46e6f5207210.PNG) + +errorlog: + +``` +kaggle.rest.ApiException: (403) +Reason: Forbidden +HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Thu, 23 Jun 2022 11:31:18 GMT', 'Access-Control-Allow-Credentials': 'true', 'Set-Cookie': 'ka_sessionid=6817a347c75399a531148e19cad0aaeb; max-age=2626560; path=/, GCLB=CIGths3--ebbUg; path=/; HttpOnly', 'Transfer-Encoding': 'chunked', 'Vary': +HTTP response body: b'{"code":403,"message":"You must accept this competition\\u0027s rules before you\\u0027ll be able to download files."}' + +``` +This error occours for two reasons: +- Your Kaggle account is not verified with your phone number. +- Rules for this specific competitions are not accepted. + +Lets accept Rules of Bulldozers competition +![kaggle-secret-error-02](https://user-images.githubusercontent.com/17012391/175291406-7a30e06d-fc05-44c3-b33c-bccd31b381bd.PNG) + +Click on "I Understand and Accept". After this you will be prompted to verify your account using your phone number: +![kaggle-secret-error-03](https://user-images.githubusercontent.com/17012391/175291608-daad1a47-119a-4e47-b48b-4f878d65ddd7.PNG) + +Add your phone number and Kaggle will send the code to your number, enter this code and verify your account. ( Note: pipeline wont run if your Kaggle account is not verified ) + +## Success +After the kaggle account is verified pipeline run is successful we will get the following: + +Screenshot 2022-06-10 at 12 04 48 AM + + diff --git a/bluebook-for-bulldozers-kaggle-competition/blue-book-for-bulldozers-kaggle-competition-kfp.py b/bluebook-for-bulldozers-kaggle-competition/blue-book-for-bulldozers-kaggle-competition-kfp.py index fc0f6152..1ed7174a 100644 --- a/bluebook-for-bulldozers-kaggle-competition/blue-book-for-bulldozers-kaggle-competition-kfp.py +++ b/bluebook-for-bulldozers-kaggle-competition/blue-book-for-bulldozers-kaggle-competition-kfp.py @@ -1,62 +1,62 @@ -import kfp -from kfp import dsl - -def LoadData(): - vop = dsl.VolumeOp(name="pvc", - resource_name="pvc", size='1Gi', - modes=dsl.VOLUME_MODE_RWO) - - return dsl.ContainerOp( - name = 'load-data', - image = 'hubdocker76/bulldozers:v6', - command = ['python3', 'load.py'], - - pvolumes={ - '/data': vop.volume - } - ) - -def PreProcess(comp1): - return dsl.ContainerOp( - name = 'preprocess', - image = 'hubdocker76/bulldozers-preprocess:v1', - pvolumes={ - '/data': comp1.pvolumes['/data'] - }, - command = ['python3', 'preprocess.py'] - ) - -def Train(comp2): - return dsl.ContainerOp( - name = 'train', - image = 'hubdocker76/bulldozers-train:v2', - pvolumes={ - '/data': comp2.pvolumes['/data'] - }, - command = ['python3', 'train.py'] - ) - -def Test(comp3): - return dsl.ContainerOp( - name = 'test', - image = 'hubdocker76/bulldozers-test:v2', - pvolumes={ - '/data': comp3.pvolumes['/data'] - }, - command = ['python3', 'test.py'] - ) - - -@dsl.pipeline( - name = 'blue book for bulldozers', - description = 'pipeline to run blue book for bulldozers') - -def passing_parameter(): - comp1 = LoadData().add_pod_label("kaggle-secret", "true") - comp2 = PreProcess(comp1) - comp3 = Train(comp2) - comp4 = Test(comp3) - -if __name__ == '__main__': - import kfp.compiler as compiler - compiler.Compiler().compile(passing_parameter, __file__[:-3]+ '.yaml') +import kfp +from kfp import dsl + +def LoadData(): + vop = dsl.VolumeOp(name="pvc", + resource_name="pvc", size='1Gi', + modes=dsl.VOLUME_MODE_RWO) + + return dsl.ContainerOp( + name = 'load-data', + image = 'hubdocker76/bulldozers:v6', + command = ['python3', 'load.py'], + + pvolumes={ + '/data': vop.volume + } + ) + +def PreProcess(comp1): + return dsl.ContainerOp( + name = 'preprocess', + image = 'hubdocker76/bulldozers-preprocess:v1', + pvolumes={ + '/data': comp1.pvolumes['/data'] + }, + command = ['python3', 'preprocess.py'] + ) + +def Train(comp2): + return dsl.ContainerOp( + name = 'train', + image = 'hubdocker76/bulldozers-train:v2', + pvolumes={ + '/data': comp2.pvolumes['/data'] + }, + command = ['python3', 'train.py'] + ) + +def Test(comp3): + return dsl.ContainerOp( + name = 'test', + image = 'hubdocker76/bulldozers-test:v2', + pvolumes={ + '/data': comp3.pvolumes['/data'] + }, + command = ['python3', 'test.py'] + ) + + +@dsl.pipeline( + name = 'blue book for bulldozers', + description = 'pipeline to run blue book for bulldozers') + +def passing_parameter(): + comp1 = LoadData().add_pod_label("kaggle-secret", "true") + comp2 = PreProcess(comp1) + comp3 = Train(comp2) + comp4 = Test(comp3) + +if __name__ == '__main__': + import kfp.compiler as compiler + compiler.Compiler().compile(passing_parameter, __file__[:-3]+ '.yaml') diff --git a/code_search/docker/ks/launch_search_index_creator_job.sh b/code_search/docker/ks/launch_search_index_creator_job.sh old mode 100755 new mode 100644 diff --git a/code_search/docker/ks/submit_code_embeddings_job.sh b/code_search/docker/ks/submit_code_embeddings_job.sh old mode 100755 new mode 100644 diff --git a/code_search/docker/ks/update_index.sh b/code_search/docker/ks/update_index.sh old mode 100755 new mode 100644 diff --git a/code_search/docker/t2t/t2t-entrypoint.sh b/code_search/docker/t2t/t2t-entrypoint.sh old mode 100755 new mode 100644 diff --git a/code_search/docker/ui/build.sh b/code_search/docker/ui/build.sh old mode 100755 new mode 100644 diff --git a/code_search/kubeflow/environments/base.libsonnet b/code_search/kubeflow/environments/base.libsonnet deleted file mode 100644 index a129affb..00000000 --- a/code_search/kubeflow/environments/base.libsonnet +++ /dev/null @@ -1,4 +0,0 @@ -local components = std.extVar("__ksonnet/components"); -components + { - // Insert user-specified overrides here. -} diff --git a/code_search/kubeflow/environments/cs_demo/globals.libsonnet b/code_search/kubeflow/environments/cs_demo/globals.libsonnet deleted file mode 100644 index 7055aebe..00000000 --- a/code_search/kubeflow/environments/cs_demo/globals.libsonnet +++ /dev/null @@ -1,9 +0,0 @@ -{ - // Warning: Do not define a global "image" as that will end up overriding - // the image parameter for all components. Define more specific names - // e.g. "dataflowImage", "trainerCpuImage", "trainerGpuImage", - workingDir: "gs://code-search-demo/20181104", - dataDir: "gs://code-search-demo/20181104/data", - project: "code-search-demo", - experiment: "demo-trainer-11-07-dist-sync-gpu", -} diff --git a/code_search/kubeflow/environments/cs_demo/main.jsonnet b/code_search/kubeflow/environments/cs_demo/main.jsonnet deleted file mode 100644 index 1a44c481..00000000 --- a/code_search/kubeflow/environments/cs_demo/main.jsonnet +++ /dev/null @@ -1,8 +0,0 @@ -local base = import "base.libsonnet"; -// uncomment if you reference ksonnet-lib -// local k = import "k.libsonnet"; - -base { - // Insert user-specified overrides here. For example if a component is named \"nginx-deployment\", you might have something like:\n") - // "nginx-deployment"+: k.deployment.mixin.metadata.labels({foo: "bar"}) -} diff --git a/code_search/kubeflow/environments/cs_demo/params.libsonnet b/code_search/kubeflow/environments/cs_demo/params.libsonnet deleted file mode 100644 index e57c4be5..00000000 --- a/code_search/kubeflow/environments/cs_demo/params.libsonnet +++ /dev/null @@ -1,22 +0,0 @@ -local params = std.extVar('__ksonnet/params'); -local globals = import 'globals.libsonnet'; -local envParams = params { - components+: { - "t2t-code-search"+: {}, - "t2t-code-search-datagen"+: { - githubTable: '', - }, - "submit-preprocess-job"+: { - githubTable: '', - }, - "search-index-server"+: { - }, - }, -}; - -{ - components: { - [x]: envParams.components[x] + globals - for x in std.objectFields(envParams.components) - }, -} \ No newline at end of file diff --git a/code_search/kubeflow/environments/pipeline/globals.libsonnet b/code_search/kubeflow/environments/pipeline/globals.libsonnet deleted file mode 100644 index 1b59385f..00000000 --- a/code_search/kubeflow/environments/pipeline/globals.libsonnet +++ /dev/null @@ -1,7 +0,0 @@ -{ - // Warning: Do not define a global "image" as that will end up overriding - // the image parameter for all components. Define more specific names - // e.g. "dataflowImage", "trainerCpuImage", "trainerGpuImage", - experiment: "pipeline", - waitUntilFinish: "true", -} diff --git a/code_search/kubeflow/environments/pipeline/main.jsonnet b/code_search/kubeflow/environments/pipeline/main.jsonnet deleted file mode 100644 index 58695a80..00000000 --- a/code_search/kubeflow/environments/pipeline/main.jsonnet +++ /dev/null @@ -1,8 +0,0 @@ -local base = import "base.libsonnet"; -// uncomment if you reference ksonnet-lib -// local k = import "k.libsonnet"; - -base + { - // Insert user-specified overrides here. For example if a component is named \"nginx-deployment\", you might have something like:\n") - // "nginx-deployment"+: k.deployment.mixin.metadata.labels({foo: "bar"}) -} diff --git a/code_search/kubeflow/environments/pipeline/params.libsonnet b/code_search/kubeflow/environments/pipeline/params.libsonnet deleted file mode 100644 index eb3bac70..00000000 --- a/code_search/kubeflow/environments/pipeline/params.libsonnet +++ /dev/null @@ -1,12 +0,0 @@ -local params = std.extVar("__ksonnet/params"); -local globals = import "globals.libsonnet"; -local envParams = params + { - components +: { - }, -}; - -{ - components: { - [x]: envParams.components[x] + globals, for x in std.objectFields(envParams.components) - }, -} diff --git a/code_search/src/code_search/nmslib/cli/start_test_server.sh b/code_search/src/code_search/nmslib/cli/start_test_server.sh old mode 100755 new mode 100644 diff --git a/codes/volume_parallel.py b/codes/volume_parallel.py index 988212e3..ee9c8910 100644 --- a/codes/volume_parallel.py +++ b/codes/volume_parallel.py @@ -1,62 +1,62 @@ - -import kfp -from kfp import dsl - -def create_pv(): - return dsl.VolumeOp( - name="create_pv", - resource_name="kfp-pvc", - size="1Gi", - modes=dsl.VOLUME_MODE_RWO - ) - - -def parallel_1(vol_name: str): - cop = dsl.ContainerOp( - name='generate_data', - image='bash:5.1', - command=['sh', '-c'], - arguments=['echo 1 | tee /mnt/out1.txt'] - ) - cop.container.set_image_pull_policy('IfNotPresent') - cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) - return cop - - -def parallel_2(vol_name: str): - cop = dsl.ContainerOp( - name='generate_data', - image='bash:5.1', - command=['sh', '-c'], - arguments=['echo 2 | tee /mnt/out2.txt'] - ) - cop.container.set_image_pull_policy('IfNotPresent') - cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) - return cop - - -def parallel_3(vol_name: str): - cop = dsl.ContainerOp( - name='generate_data', - image='bash:5.1', - command=['sh', '-c'], - arguments=['echo 3 | tee /mnt/out3.txt'] - ) - cop.container.set_image_pull_policy('IfNotPresent') - cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) - return cop - - -@dsl.pipeline( - name="Kubeflow volume parallel example", - description="Demonstrate the use case of volume on Kubeflow pipeline.") -def volume_parallel(): - vop = create_pv() - cop1 = parallel_1(vop.outputs["name"]).after(vop) - cop2 = parallel_2(vop.outputs["name"]).after(vop) - cop3 = parallel_3(vop.outputs["name"]).after(vop) - - -if __name__ == "__main__": - import kfp.compiler as compiler + +import kfp +from kfp import dsl + +def create_pv(): + return dsl.VolumeOp( + name="create_pv", + resource_name="kfp-pvc", + size="1Gi", + modes=dsl.VOLUME_MODE_RWO + ) + + +def parallel_1(vol_name: str): + cop = dsl.ContainerOp( + name='generate_data', + image='bash:5.1', + command=['sh', '-c'], + arguments=['echo 1 | tee /mnt/out1.txt'] + ) + cop.container.set_image_pull_policy('IfNotPresent') + cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) + return cop + + +def parallel_2(vol_name: str): + cop = dsl.ContainerOp( + name='generate_data', + image='bash:5.1', + command=['sh', '-c'], + arguments=['echo 2 | tee /mnt/out2.txt'] + ) + cop.container.set_image_pull_policy('IfNotPresent') + cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) + return cop + + +def parallel_3(vol_name: str): + cop = dsl.ContainerOp( + name='generate_data', + image='bash:5.1', + command=['sh', '-c'], + arguments=['echo 3 | tee /mnt/out3.txt'] + ) + cop.container.set_image_pull_policy('IfNotPresent') + cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) + return cop + + +@dsl.pipeline( + name="Kubeflow volume parallel example", + description="Demonstrate the use case of volume on Kubeflow pipeline.") +def volume_parallel(): + vop = create_pv() + cop1 = parallel_1(vop.outputs["name"]).after(vop) + cop2 = parallel_2(vop.outputs["name"]).after(vop) + cop3 = parallel_3(vop.outputs["name"]).after(vop) + + +if __name__ == "__main__": + import kfp.compiler as compiler compiler.Compiler().compile(volume_parallel, __file__ + ".yaml") \ No newline at end of file diff --git a/demos/simple_pipeline/gpu-example-pipeline.py b/demos/simple_pipeline/gpu-example-pipeline.py old mode 100755 new mode 100644 diff --git a/demos/yelp_demo/demo_setup/create_context.sh b/demos/yelp_demo/demo_setup/create_context.sh old mode 100755 new mode 100644 diff --git a/demos/yelp_demo/pipelines/gpu-example-pipeline.py b/demos/yelp_demo/pipelines/gpu-example-pipeline.py old mode 100755 new mode 100644 diff --git a/demos/yelp_demo/yelp/yelp_sentiment/worker_launcher.sh b/demos/yelp_demo/yelp/yelp_sentiment/worker_launcher.sh old mode 100755 new mode 100644 diff --git a/digit-recognition-kaggle-competition/data/sample_submission.csv b/digit-recognition-kaggle-competition/data/sample_submission.csv index f5becf71..7ea007bb 100644 --- a/digit-recognition-kaggle-competition/data/sample_submission.csv +++ b/digit-recognition-kaggle-competition/data/sample_submission.csv @@ -1,28001 +1,28001 @@ -ImageId,Label -1,0 -2,0 -3,0 -4,0 -5,0 -6,0 -7,0 -8,0 -9,0 -10,0 -11,0 -12,0 -13,0 -14,0 -15,0 -16,0 -17,0 -18,0 -19,0 -20,0 -21,0 -22,0 -23,0 -24,0 -25,0 -26,0 -27,0 -28,0 -29,0 -30,0 -31,0 -32,0 -33,0 -34,0 -35,0 -36,0 -37,0 -38,0 -39,0 -40,0 -41,0 -42,0 -43,0 -44,0 -45,0 -46,0 -47,0 -48,0 -49,0 -50,0 -51,0 -52,0 -53,0 -54,0 -55,0 -56,0 -57,0 -58,0 -59,0 -60,0 -61,0 -62,0 -63,0 -64,0 -65,0 -66,0 -67,0 -68,0 -69,0 -70,0 -71,0 -72,0 -73,0 -74,0 -75,0 -76,0 -77,0 -78,0 -79,0 -80,0 -81,0 -82,0 -83,0 -84,0 -85,0 -86,0 -87,0 -88,0 -89,0 -90,0 -91,0 -92,0 -93,0 -94,0 -95,0 -96,0 -97,0 -98,0 -99,0 -100,0 -101,0 -102,0 -103,0 -104,0 -105,0 -106,0 -107,0 -108,0 -109,0 -110,0 -111,0 -112,0 -113,0 -114,0 -115,0 -116,0 -117,0 -118,0 -119,0 -120,0 -121,0 -122,0 -123,0 -124,0 -125,0 -126,0 -127,0 -128,0 -129,0 -130,0 -131,0 -132,0 -133,0 -134,0 -135,0 -136,0 -137,0 -138,0 -139,0 -140,0 -141,0 -142,0 -143,0 -144,0 -145,0 -146,0 -147,0 -148,0 -149,0 -150,0 -151,0 -152,0 -153,0 -154,0 -155,0 -156,0 -157,0 -158,0 -159,0 -160,0 -161,0 -162,0 -163,0 -164,0 -165,0 -166,0 -167,0 -168,0 -169,0 -170,0 -171,0 -172,0 -173,0 -174,0 -175,0 -176,0 -177,0 -178,0 -179,0 -180,0 -181,0 -182,0 -183,0 -184,0 -185,0 -186,0 -187,0 -188,0 -189,0 -190,0 -191,0 -192,0 -193,0 -194,0 -195,0 -196,0 -197,0 -198,0 -199,0 -200,0 -201,0 -202,0 -203,0 -204,0 -205,0 -206,0 -207,0 -208,0 -209,0 -210,0 -211,0 -212,0 -213,0 -214,0 -215,0 -216,0 -217,0 -218,0 -219,0 -220,0 -221,0 -222,0 -223,0 -224,0 -225,0 -226,0 -227,0 -228,0 -229,0 -230,0 -231,0 -232,0 -233,0 -234,0 -235,0 -236,0 -237,0 -238,0 -239,0 -240,0 -241,0 -242,0 -243,0 -244,0 -245,0 -246,0 -247,0 -248,0 -249,0 -250,0 -251,0 -252,0 -253,0 -254,0 -255,0 -256,0 -257,0 -258,0 -259,0 -260,0 -261,0 -262,0 -263,0 -264,0 -265,0 -266,0 -267,0 -268,0 -269,0 -270,0 -271,0 -272,0 -273,0 -274,0 -275,0 -276,0 -277,0 -278,0 -279,0 -280,0 -281,0 -282,0 -283,0 -284,0 -285,0 -286,0 -287,0 -288,0 -289,0 -290,0 -291,0 -292,0 -293,0 -294,0 -295,0 -296,0 -297,0 -298,0 -299,0 -300,0 -301,0 -302,0 -303,0 -304,0 -305,0 -306,0 -307,0 -308,0 -309,0 -310,0 -311,0 -312,0 -313,0 -314,0 -315,0 -316,0 -317,0 -318,0 -319,0 -320,0 -321,0 -322,0 -323,0 -324,0 -325,0 -326,0 -327,0 -328,0 -329,0 -330,0 -331,0 -332,0 -333,0 -334,0 -335,0 -336,0 -337,0 -338,0 -339,0 -340,0 -341,0 -342,0 -343,0 -344,0 -345,0 -346,0 -347,0 -348,0 -349,0 -350,0 -351,0 -352,0 -353,0 -354,0 -355,0 -356,0 -357,0 -358,0 -359,0 -360,0 -361,0 -362,0 -363,0 -364,0 -365,0 -366,0 -367,0 -368,0 -369,0 -370,0 -371,0 -372,0 -373,0 -374,0 -375,0 -376,0 -377,0 -378,0 -379,0 -380,0 -381,0 -382,0 -383,0 -384,0 -385,0 -386,0 -387,0 -388,0 -389,0 -390,0 -391,0 -392,0 -393,0 -394,0 -395,0 -396,0 -397,0 -398,0 -399,0 -400,0 -401,0 -402,0 -403,0 -404,0 -405,0 -406,0 -407,0 -408,0 -409,0 -410,0 -411,0 -412,0 -413,0 -414,0 -415,0 -416,0 -417,0 -418,0 -419,0 -420,0 -421,0 -422,0 -423,0 -424,0 -425,0 -426,0 -427,0 -428,0 -429,0 -430,0 -431,0 -432,0 -433,0 -434,0 -435,0 -436,0 -437,0 -438,0 -439,0 -440,0 -441,0 -442,0 -443,0 -444,0 -445,0 -446,0 -447,0 -448,0 -449,0 -450,0 -451,0 -452,0 -453,0 -454,0 -455,0 -456,0 -457,0 -458,0 -459,0 -460,0 -461,0 -462,0 -463,0 -464,0 -465,0 -466,0 -467,0 -468,0 -469,0 -470,0 -471,0 -472,0 -473,0 -474,0 -475,0 -476,0 -477,0 -478,0 -479,0 -480,0 -481,0 -482,0 -483,0 -484,0 -485,0 -486,0 -487,0 -488,0 -489,0 -490,0 -491,0 -492,0 -493,0 -494,0 -495,0 -496,0 -497,0 -498,0 -499,0 -500,0 -501,0 -502,0 -503,0 -504,0 -505,0 -506,0 -507,0 -508,0 -509,0 -510,0 -511,0 -512,0 -513,0 -514,0 -515,0 -516,0 -517,0 -518,0 -519,0 -520,0 -521,0 -522,0 -523,0 -524,0 -525,0 -526,0 -527,0 -528,0 -529,0 -530,0 -531,0 -532,0 -533,0 -534,0 -535,0 -536,0 -537,0 -538,0 -539,0 -540,0 -541,0 -542,0 -543,0 -544,0 -545,0 -546,0 -547,0 -548,0 -549,0 -550,0 -551,0 -552,0 -553,0 -554,0 -555,0 -556,0 -557,0 -558,0 -559,0 -560,0 -561,0 -562,0 -563,0 -564,0 -565,0 -566,0 -567,0 -568,0 -569,0 -570,0 -571,0 -572,0 -573,0 -574,0 -575,0 -576,0 -577,0 -578,0 -579,0 -580,0 -581,0 -582,0 -583,0 -584,0 -585,0 -586,0 -587,0 -588,0 -589,0 -590,0 -591,0 -592,0 -593,0 -594,0 -595,0 -596,0 -597,0 -598,0 -599,0 -600,0 -601,0 -602,0 -603,0 -604,0 -605,0 -606,0 -607,0 -608,0 -609,0 -610,0 -611,0 -612,0 -613,0 -614,0 -615,0 -616,0 -617,0 -618,0 -619,0 -620,0 -621,0 -622,0 -623,0 -624,0 -625,0 -626,0 -627,0 -628,0 -629,0 -630,0 -631,0 -632,0 -633,0 -634,0 -635,0 -636,0 -637,0 -638,0 -639,0 -640,0 -641,0 -642,0 -643,0 -644,0 -645,0 -646,0 -647,0 -648,0 -649,0 -650,0 -651,0 -652,0 -653,0 -654,0 -655,0 -656,0 -657,0 -658,0 -659,0 -660,0 -661,0 -662,0 -663,0 -664,0 -665,0 -666,0 -667,0 -668,0 -669,0 -670,0 -671,0 -672,0 -673,0 -674,0 -675,0 -676,0 -677,0 -678,0 -679,0 -680,0 -681,0 -682,0 -683,0 -684,0 -685,0 -686,0 -687,0 -688,0 -689,0 -690,0 -691,0 -692,0 -693,0 -694,0 -695,0 -696,0 -697,0 -698,0 -699,0 -700,0 -701,0 -702,0 -703,0 -704,0 -705,0 -706,0 -707,0 -708,0 -709,0 -710,0 -711,0 -712,0 -713,0 -714,0 -715,0 -716,0 -717,0 -718,0 -719,0 -720,0 -721,0 -722,0 -723,0 -724,0 -725,0 -726,0 -727,0 -728,0 -729,0 -730,0 -731,0 -732,0 -733,0 -734,0 -735,0 -736,0 -737,0 -738,0 -739,0 -740,0 -741,0 -742,0 -743,0 -744,0 -745,0 -746,0 -747,0 -748,0 -749,0 -750,0 -751,0 -752,0 -753,0 -754,0 -755,0 -756,0 -757,0 -758,0 -759,0 -760,0 -761,0 -762,0 -763,0 -764,0 -765,0 -766,0 -767,0 -768,0 -769,0 -770,0 -771,0 -772,0 -773,0 -774,0 -775,0 -776,0 -777,0 -778,0 -779,0 -780,0 -781,0 -782,0 -783,0 -784,0 -785,0 -786,0 -787,0 -788,0 -789,0 -790,0 -791,0 -792,0 -793,0 -794,0 -795,0 -796,0 -797,0 -798,0 -799,0 -800,0 -801,0 -802,0 -803,0 -804,0 -805,0 -806,0 -807,0 -808,0 -809,0 -810,0 -811,0 -812,0 -813,0 -814,0 -815,0 -816,0 -817,0 -818,0 -819,0 -820,0 -821,0 -822,0 -823,0 -824,0 -825,0 -826,0 -827,0 -828,0 -829,0 -830,0 -831,0 -832,0 -833,0 -834,0 -835,0 -836,0 -837,0 -838,0 -839,0 -840,0 -841,0 -842,0 -843,0 -844,0 -845,0 -846,0 -847,0 -848,0 -849,0 -850,0 -851,0 -852,0 -853,0 -854,0 -855,0 -856,0 -857,0 -858,0 -859,0 -860,0 -861,0 -862,0 -863,0 -864,0 -865,0 -866,0 -867,0 -868,0 -869,0 -870,0 -871,0 -872,0 -873,0 -874,0 -875,0 -876,0 -877,0 -878,0 -879,0 -880,0 -881,0 -882,0 -883,0 -884,0 -885,0 -886,0 -887,0 -888,0 -889,0 -890,0 -891,0 -892,0 -893,0 -894,0 -895,0 -896,0 -897,0 -898,0 -899,0 -900,0 -901,0 -902,0 -903,0 -904,0 -905,0 -906,0 -907,0 -908,0 -909,0 -910,0 -911,0 -912,0 -913,0 -914,0 -915,0 -916,0 -917,0 -918,0 -919,0 -920,0 -921,0 -922,0 -923,0 -924,0 -925,0 -926,0 -927,0 -928,0 -929,0 -930,0 -931,0 -932,0 -933,0 -934,0 -935,0 -936,0 -937,0 -938,0 -939,0 -940,0 -941,0 -942,0 -943,0 -944,0 -945,0 -946,0 -947,0 -948,0 -949,0 -950,0 -951,0 -952,0 -953,0 -954,0 -955,0 -956,0 -957,0 -958,0 -959,0 -960,0 -961,0 -962,0 -963,0 -964,0 -965,0 -966,0 -967,0 -968,0 -969,0 -970,0 -971,0 -972,0 -973,0 -974,0 -975,0 -976,0 -977,0 -978,0 -979,0 -980,0 -981,0 -982,0 -983,0 -984,0 -985,0 -986,0 -987,0 -988,0 -989,0 -990,0 -991,0 -992,0 -993,0 -994,0 -995,0 -996,0 -997,0 -998,0 -999,0 -1000,0 -1001,0 -1002,0 -1003,0 -1004,0 -1005,0 -1006,0 -1007,0 -1008,0 -1009,0 -1010,0 -1011,0 -1012,0 -1013,0 -1014,0 -1015,0 -1016,0 -1017,0 -1018,0 -1019,0 -1020,0 -1021,0 -1022,0 -1023,0 -1024,0 -1025,0 -1026,0 -1027,0 -1028,0 -1029,0 -1030,0 -1031,0 -1032,0 -1033,0 -1034,0 -1035,0 -1036,0 -1037,0 -1038,0 -1039,0 -1040,0 -1041,0 -1042,0 -1043,0 -1044,0 -1045,0 -1046,0 -1047,0 -1048,0 -1049,0 -1050,0 -1051,0 -1052,0 -1053,0 -1054,0 -1055,0 -1056,0 -1057,0 -1058,0 -1059,0 -1060,0 -1061,0 -1062,0 -1063,0 -1064,0 -1065,0 -1066,0 -1067,0 -1068,0 -1069,0 -1070,0 -1071,0 -1072,0 -1073,0 -1074,0 -1075,0 -1076,0 -1077,0 -1078,0 -1079,0 -1080,0 -1081,0 -1082,0 -1083,0 -1084,0 -1085,0 -1086,0 -1087,0 -1088,0 -1089,0 -1090,0 -1091,0 -1092,0 -1093,0 -1094,0 -1095,0 -1096,0 -1097,0 -1098,0 -1099,0 -1100,0 -1101,0 -1102,0 -1103,0 -1104,0 -1105,0 -1106,0 -1107,0 -1108,0 -1109,0 -1110,0 -1111,0 -1112,0 -1113,0 -1114,0 -1115,0 -1116,0 -1117,0 -1118,0 -1119,0 -1120,0 -1121,0 -1122,0 -1123,0 -1124,0 -1125,0 -1126,0 -1127,0 -1128,0 -1129,0 -1130,0 -1131,0 -1132,0 -1133,0 -1134,0 -1135,0 -1136,0 -1137,0 -1138,0 -1139,0 -1140,0 -1141,0 -1142,0 -1143,0 -1144,0 -1145,0 -1146,0 -1147,0 -1148,0 -1149,0 -1150,0 -1151,0 -1152,0 -1153,0 -1154,0 -1155,0 -1156,0 -1157,0 -1158,0 -1159,0 -1160,0 -1161,0 -1162,0 -1163,0 -1164,0 -1165,0 -1166,0 -1167,0 -1168,0 -1169,0 -1170,0 -1171,0 -1172,0 -1173,0 -1174,0 -1175,0 -1176,0 -1177,0 -1178,0 -1179,0 -1180,0 -1181,0 -1182,0 -1183,0 -1184,0 -1185,0 -1186,0 -1187,0 -1188,0 -1189,0 -1190,0 -1191,0 -1192,0 -1193,0 -1194,0 -1195,0 -1196,0 -1197,0 -1198,0 -1199,0 -1200,0 -1201,0 -1202,0 -1203,0 -1204,0 -1205,0 -1206,0 -1207,0 -1208,0 -1209,0 -1210,0 -1211,0 -1212,0 -1213,0 -1214,0 -1215,0 -1216,0 -1217,0 -1218,0 -1219,0 -1220,0 -1221,0 -1222,0 -1223,0 -1224,0 -1225,0 -1226,0 -1227,0 -1228,0 -1229,0 -1230,0 -1231,0 -1232,0 -1233,0 -1234,0 -1235,0 -1236,0 -1237,0 -1238,0 -1239,0 -1240,0 -1241,0 -1242,0 -1243,0 -1244,0 -1245,0 -1246,0 -1247,0 -1248,0 -1249,0 -1250,0 -1251,0 -1252,0 -1253,0 -1254,0 -1255,0 -1256,0 -1257,0 -1258,0 -1259,0 -1260,0 -1261,0 -1262,0 -1263,0 -1264,0 -1265,0 -1266,0 -1267,0 -1268,0 -1269,0 -1270,0 -1271,0 -1272,0 -1273,0 -1274,0 -1275,0 -1276,0 -1277,0 -1278,0 -1279,0 -1280,0 -1281,0 -1282,0 -1283,0 -1284,0 -1285,0 -1286,0 -1287,0 -1288,0 -1289,0 -1290,0 -1291,0 -1292,0 -1293,0 -1294,0 -1295,0 -1296,0 -1297,0 -1298,0 -1299,0 -1300,0 -1301,0 -1302,0 -1303,0 -1304,0 -1305,0 -1306,0 -1307,0 -1308,0 -1309,0 -1310,0 -1311,0 -1312,0 -1313,0 -1314,0 -1315,0 -1316,0 -1317,0 -1318,0 -1319,0 -1320,0 -1321,0 -1322,0 -1323,0 -1324,0 -1325,0 -1326,0 -1327,0 -1328,0 -1329,0 -1330,0 -1331,0 -1332,0 -1333,0 -1334,0 -1335,0 -1336,0 -1337,0 -1338,0 -1339,0 -1340,0 -1341,0 -1342,0 -1343,0 -1344,0 -1345,0 -1346,0 -1347,0 -1348,0 -1349,0 -1350,0 -1351,0 -1352,0 -1353,0 -1354,0 -1355,0 -1356,0 -1357,0 -1358,0 -1359,0 -1360,0 -1361,0 -1362,0 -1363,0 -1364,0 -1365,0 -1366,0 -1367,0 -1368,0 -1369,0 -1370,0 -1371,0 -1372,0 -1373,0 -1374,0 -1375,0 -1376,0 -1377,0 -1378,0 -1379,0 -1380,0 -1381,0 -1382,0 -1383,0 -1384,0 -1385,0 -1386,0 -1387,0 -1388,0 -1389,0 -1390,0 -1391,0 -1392,0 -1393,0 -1394,0 -1395,0 -1396,0 -1397,0 -1398,0 -1399,0 -1400,0 -1401,0 -1402,0 -1403,0 -1404,0 -1405,0 -1406,0 -1407,0 -1408,0 -1409,0 -1410,0 -1411,0 -1412,0 -1413,0 -1414,0 -1415,0 -1416,0 -1417,0 -1418,0 -1419,0 -1420,0 -1421,0 -1422,0 -1423,0 -1424,0 -1425,0 -1426,0 -1427,0 -1428,0 -1429,0 -1430,0 -1431,0 -1432,0 -1433,0 -1434,0 -1435,0 -1436,0 -1437,0 -1438,0 -1439,0 -1440,0 -1441,0 -1442,0 -1443,0 -1444,0 -1445,0 -1446,0 -1447,0 -1448,0 -1449,0 -1450,0 -1451,0 -1452,0 -1453,0 -1454,0 -1455,0 -1456,0 -1457,0 -1458,0 -1459,0 -1460,0 -1461,0 -1462,0 -1463,0 -1464,0 -1465,0 -1466,0 -1467,0 -1468,0 -1469,0 -1470,0 -1471,0 -1472,0 -1473,0 -1474,0 -1475,0 -1476,0 -1477,0 -1478,0 -1479,0 -1480,0 -1481,0 -1482,0 -1483,0 -1484,0 -1485,0 -1486,0 -1487,0 -1488,0 -1489,0 -1490,0 -1491,0 -1492,0 -1493,0 -1494,0 -1495,0 -1496,0 -1497,0 -1498,0 -1499,0 -1500,0 -1501,0 -1502,0 -1503,0 -1504,0 -1505,0 -1506,0 -1507,0 -1508,0 -1509,0 -1510,0 -1511,0 -1512,0 -1513,0 -1514,0 -1515,0 -1516,0 -1517,0 -1518,0 -1519,0 -1520,0 -1521,0 -1522,0 -1523,0 -1524,0 -1525,0 -1526,0 -1527,0 -1528,0 -1529,0 -1530,0 -1531,0 -1532,0 -1533,0 -1534,0 -1535,0 -1536,0 -1537,0 -1538,0 -1539,0 -1540,0 -1541,0 -1542,0 -1543,0 -1544,0 -1545,0 -1546,0 -1547,0 -1548,0 -1549,0 -1550,0 -1551,0 -1552,0 -1553,0 -1554,0 -1555,0 -1556,0 -1557,0 -1558,0 -1559,0 -1560,0 -1561,0 -1562,0 -1563,0 -1564,0 -1565,0 -1566,0 -1567,0 -1568,0 -1569,0 -1570,0 -1571,0 -1572,0 -1573,0 -1574,0 -1575,0 -1576,0 -1577,0 -1578,0 -1579,0 -1580,0 -1581,0 -1582,0 -1583,0 -1584,0 -1585,0 -1586,0 -1587,0 -1588,0 -1589,0 -1590,0 -1591,0 -1592,0 -1593,0 -1594,0 -1595,0 -1596,0 -1597,0 -1598,0 -1599,0 -1600,0 -1601,0 -1602,0 -1603,0 -1604,0 -1605,0 -1606,0 -1607,0 -1608,0 -1609,0 -1610,0 -1611,0 -1612,0 -1613,0 -1614,0 -1615,0 -1616,0 -1617,0 -1618,0 -1619,0 -1620,0 -1621,0 -1622,0 -1623,0 -1624,0 -1625,0 -1626,0 -1627,0 -1628,0 -1629,0 -1630,0 -1631,0 -1632,0 -1633,0 -1634,0 -1635,0 -1636,0 -1637,0 -1638,0 -1639,0 -1640,0 -1641,0 -1642,0 -1643,0 -1644,0 -1645,0 -1646,0 -1647,0 -1648,0 -1649,0 -1650,0 -1651,0 -1652,0 -1653,0 -1654,0 -1655,0 -1656,0 -1657,0 -1658,0 -1659,0 -1660,0 -1661,0 -1662,0 -1663,0 -1664,0 -1665,0 -1666,0 -1667,0 -1668,0 -1669,0 -1670,0 -1671,0 -1672,0 -1673,0 -1674,0 -1675,0 -1676,0 -1677,0 -1678,0 -1679,0 -1680,0 -1681,0 -1682,0 -1683,0 -1684,0 -1685,0 -1686,0 -1687,0 -1688,0 -1689,0 -1690,0 -1691,0 -1692,0 -1693,0 -1694,0 -1695,0 -1696,0 -1697,0 -1698,0 -1699,0 -1700,0 -1701,0 -1702,0 -1703,0 -1704,0 -1705,0 -1706,0 -1707,0 -1708,0 -1709,0 -1710,0 -1711,0 -1712,0 -1713,0 -1714,0 -1715,0 -1716,0 -1717,0 -1718,0 -1719,0 -1720,0 -1721,0 -1722,0 -1723,0 -1724,0 -1725,0 -1726,0 -1727,0 -1728,0 -1729,0 -1730,0 -1731,0 -1732,0 -1733,0 -1734,0 -1735,0 -1736,0 -1737,0 -1738,0 -1739,0 -1740,0 -1741,0 -1742,0 -1743,0 -1744,0 -1745,0 -1746,0 -1747,0 -1748,0 -1749,0 -1750,0 -1751,0 -1752,0 -1753,0 -1754,0 -1755,0 -1756,0 -1757,0 -1758,0 -1759,0 -1760,0 -1761,0 -1762,0 -1763,0 -1764,0 -1765,0 -1766,0 -1767,0 -1768,0 -1769,0 -1770,0 -1771,0 -1772,0 -1773,0 -1774,0 -1775,0 -1776,0 -1777,0 -1778,0 -1779,0 -1780,0 -1781,0 -1782,0 -1783,0 -1784,0 -1785,0 -1786,0 -1787,0 -1788,0 -1789,0 -1790,0 -1791,0 -1792,0 -1793,0 -1794,0 -1795,0 -1796,0 -1797,0 -1798,0 -1799,0 -1800,0 -1801,0 -1802,0 -1803,0 -1804,0 -1805,0 -1806,0 -1807,0 -1808,0 -1809,0 -1810,0 -1811,0 -1812,0 -1813,0 -1814,0 -1815,0 -1816,0 -1817,0 -1818,0 -1819,0 -1820,0 -1821,0 -1822,0 -1823,0 -1824,0 -1825,0 -1826,0 -1827,0 -1828,0 -1829,0 -1830,0 -1831,0 -1832,0 -1833,0 -1834,0 -1835,0 -1836,0 -1837,0 -1838,0 -1839,0 -1840,0 -1841,0 -1842,0 -1843,0 -1844,0 -1845,0 -1846,0 -1847,0 -1848,0 -1849,0 -1850,0 -1851,0 -1852,0 -1853,0 -1854,0 -1855,0 -1856,0 -1857,0 -1858,0 -1859,0 -1860,0 -1861,0 -1862,0 -1863,0 -1864,0 -1865,0 -1866,0 -1867,0 -1868,0 -1869,0 -1870,0 -1871,0 -1872,0 -1873,0 -1874,0 -1875,0 -1876,0 -1877,0 -1878,0 -1879,0 -1880,0 -1881,0 -1882,0 -1883,0 -1884,0 -1885,0 -1886,0 -1887,0 -1888,0 -1889,0 -1890,0 -1891,0 -1892,0 -1893,0 -1894,0 -1895,0 -1896,0 -1897,0 -1898,0 -1899,0 -1900,0 -1901,0 -1902,0 -1903,0 -1904,0 -1905,0 -1906,0 -1907,0 -1908,0 -1909,0 -1910,0 -1911,0 -1912,0 -1913,0 -1914,0 -1915,0 -1916,0 -1917,0 -1918,0 -1919,0 -1920,0 -1921,0 -1922,0 -1923,0 -1924,0 -1925,0 -1926,0 -1927,0 -1928,0 -1929,0 -1930,0 -1931,0 -1932,0 -1933,0 -1934,0 -1935,0 -1936,0 -1937,0 -1938,0 -1939,0 -1940,0 -1941,0 -1942,0 -1943,0 -1944,0 -1945,0 -1946,0 -1947,0 -1948,0 -1949,0 -1950,0 -1951,0 -1952,0 -1953,0 -1954,0 -1955,0 -1956,0 -1957,0 -1958,0 -1959,0 -1960,0 -1961,0 -1962,0 -1963,0 -1964,0 -1965,0 -1966,0 -1967,0 -1968,0 -1969,0 -1970,0 -1971,0 -1972,0 -1973,0 -1974,0 -1975,0 -1976,0 -1977,0 -1978,0 -1979,0 -1980,0 -1981,0 -1982,0 -1983,0 -1984,0 -1985,0 -1986,0 -1987,0 -1988,0 -1989,0 -1990,0 -1991,0 -1992,0 -1993,0 -1994,0 -1995,0 -1996,0 -1997,0 -1998,0 -1999,0 -2000,0 -2001,0 -2002,0 -2003,0 -2004,0 -2005,0 -2006,0 -2007,0 -2008,0 -2009,0 -2010,0 -2011,0 -2012,0 -2013,0 -2014,0 -2015,0 -2016,0 -2017,0 -2018,0 -2019,0 -2020,0 -2021,0 -2022,0 -2023,0 -2024,0 -2025,0 -2026,0 -2027,0 -2028,0 -2029,0 -2030,0 -2031,0 -2032,0 -2033,0 -2034,0 -2035,0 -2036,0 -2037,0 -2038,0 -2039,0 -2040,0 -2041,0 -2042,0 -2043,0 -2044,0 -2045,0 -2046,0 -2047,0 -2048,0 -2049,0 -2050,0 -2051,0 -2052,0 -2053,0 -2054,0 -2055,0 -2056,0 -2057,0 -2058,0 -2059,0 -2060,0 -2061,0 -2062,0 -2063,0 -2064,0 -2065,0 -2066,0 -2067,0 -2068,0 -2069,0 -2070,0 -2071,0 -2072,0 -2073,0 -2074,0 -2075,0 -2076,0 -2077,0 -2078,0 -2079,0 -2080,0 -2081,0 -2082,0 -2083,0 -2084,0 -2085,0 -2086,0 -2087,0 -2088,0 -2089,0 -2090,0 -2091,0 -2092,0 -2093,0 -2094,0 -2095,0 -2096,0 -2097,0 -2098,0 -2099,0 -2100,0 -2101,0 -2102,0 -2103,0 -2104,0 -2105,0 -2106,0 -2107,0 -2108,0 -2109,0 -2110,0 -2111,0 -2112,0 -2113,0 -2114,0 -2115,0 -2116,0 -2117,0 -2118,0 -2119,0 -2120,0 -2121,0 -2122,0 -2123,0 -2124,0 -2125,0 -2126,0 -2127,0 -2128,0 -2129,0 -2130,0 -2131,0 -2132,0 -2133,0 -2134,0 -2135,0 -2136,0 -2137,0 -2138,0 -2139,0 -2140,0 -2141,0 -2142,0 -2143,0 -2144,0 -2145,0 -2146,0 -2147,0 -2148,0 -2149,0 -2150,0 -2151,0 -2152,0 -2153,0 -2154,0 -2155,0 -2156,0 -2157,0 -2158,0 -2159,0 -2160,0 -2161,0 -2162,0 -2163,0 -2164,0 -2165,0 -2166,0 -2167,0 -2168,0 -2169,0 -2170,0 -2171,0 -2172,0 -2173,0 -2174,0 -2175,0 -2176,0 -2177,0 -2178,0 -2179,0 -2180,0 -2181,0 -2182,0 -2183,0 -2184,0 -2185,0 -2186,0 -2187,0 -2188,0 -2189,0 -2190,0 -2191,0 -2192,0 -2193,0 -2194,0 -2195,0 -2196,0 -2197,0 -2198,0 -2199,0 -2200,0 -2201,0 -2202,0 -2203,0 -2204,0 -2205,0 -2206,0 -2207,0 -2208,0 -2209,0 -2210,0 -2211,0 -2212,0 -2213,0 -2214,0 -2215,0 -2216,0 -2217,0 -2218,0 -2219,0 -2220,0 -2221,0 -2222,0 -2223,0 -2224,0 -2225,0 -2226,0 -2227,0 -2228,0 -2229,0 -2230,0 -2231,0 -2232,0 -2233,0 -2234,0 -2235,0 -2236,0 -2237,0 -2238,0 -2239,0 -2240,0 -2241,0 -2242,0 -2243,0 -2244,0 -2245,0 -2246,0 -2247,0 -2248,0 -2249,0 -2250,0 -2251,0 -2252,0 -2253,0 -2254,0 -2255,0 -2256,0 -2257,0 -2258,0 -2259,0 -2260,0 -2261,0 -2262,0 -2263,0 -2264,0 -2265,0 -2266,0 -2267,0 -2268,0 -2269,0 -2270,0 -2271,0 -2272,0 -2273,0 -2274,0 -2275,0 -2276,0 -2277,0 -2278,0 -2279,0 -2280,0 -2281,0 -2282,0 -2283,0 -2284,0 -2285,0 -2286,0 -2287,0 -2288,0 -2289,0 -2290,0 -2291,0 -2292,0 -2293,0 -2294,0 -2295,0 -2296,0 -2297,0 -2298,0 -2299,0 -2300,0 -2301,0 -2302,0 -2303,0 -2304,0 -2305,0 -2306,0 -2307,0 -2308,0 -2309,0 -2310,0 -2311,0 -2312,0 -2313,0 -2314,0 -2315,0 -2316,0 -2317,0 -2318,0 -2319,0 -2320,0 -2321,0 -2322,0 -2323,0 -2324,0 -2325,0 -2326,0 -2327,0 -2328,0 -2329,0 -2330,0 -2331,0 -2332,0 -2333,0 -2334,0 -2335,0 -2336,0 -2337,0 -2338,0 -2339,0 -2340,0 -2341,0 -2342,0 -2343,0 -2344,0 -2345,0 -2346,0 -2347,0 -2348,0 -2349,0 -2350,0 -2351,0 -2352,0 -2353,0 -2354,0 -2355,0 -2356,0 -2357,0 -2358,0 -2359,0 -2360,0 -2361,0 -2362,0 -2363,0 -2364,0 -2365,0 -2366,0 -2367,0 -2368,0 -2369,0 -2370,0 -2371,0 -2372,0 -2373,0 -2374,0 -2375,0 -2376,0 -2377,0 -2378,0 -2379,0 -2380,0 -2381,0 -2382,0 -2383,0 -2384,0 -2385,0 -2386,0 -2387,0 -2388,0 -2389,0 -2390,0 -2391,0 -2392,0 -2393,0 -2394,0 -2395,0 -2396,0 -2397,0 -2398,0 -2399,0 -2400,0 -2401,0 -2402,0 -2403,0 -2404,0 -2405,0 -2406,0 -2407,0 -2408,0 -2409,0 -2410,0 -2411,0 -2412,0 -2413,0 -2414,0 -2415,0 -2416,0 -2417,0 -2418,0 -2419,0 -2420,0 -2421,0 -2422,0 -2423,0 -2424,0 -2425,0 -2426,0 -2427,0 -2428,0 -2429,0 -2430,0 -2431,0 -2432,0 -2433,0 -2434,0 -2435,0 -2436,0 -2437,0 -2438,0 -2439,0 -2440,0 -2441,0 -2442,0 -2443,0 -2444,0 -2445,0 -2446,0 -2447,0 -2448,0 -2449,0 -2450,0 -2451,0 -2452,0 -2453,0 -2454,0 -2455,0 -2456,0 -2457,0 -2458,0 -2459,0 -2460,0 -2461,0 -2462,0 -2463,0 -2464,0 -2465,0 -2466,0 -2467,0 -2468,0 -2469,0 -2470,0 -2471,0 -2472,0 -2473,0 -2474,0 -2475,0 -2476,0 -2477,0 -2478,0 -2479,0 -2480,0 -2481,0 -2482,0 -2483,0 -2484,0 -2485,0 -2486,0 -2487,0 -2488,0 -2489,0 -2490,0 -2491,0 -2492,0 -2493,0 -2494,0 -2495,0 -2496,0 -2497,0 -2498,0 -2499,0 -2500,0 -2501,0 -2502,0 -2503,0 -2504,0 -2505,0 -2506,0 -2507,0 -2508,0 -2509,0 -2510,0 -2511,0 -2512,0 -2513,0 -2514,0 -2515,0 -2516,0 -2517,0 -2518,0 -2519,0 -2520,0 -2521,0 -2522,0 -2523,0 -2524,0 -2525,0 -2526,0 -2527,0 -2528,0 -2529,0 -2530,0 -2531,0 -2532,0 -2533,0 -2534,0 -2535,0 -2536,0 -2537,0 -2538,0 -2539,0 -2540,0 -2541,0 -2542,0 -2543,0 -2544,0 -2545,0 -2546,0 -2547,0 -2548,0 -2549,0 -2550,0 -2551,0 -2552,0 -2553,0 -2554,0 -2555,0 -2556,0 -2557,0 -2558,0 -2559,0 -2560,0 -2561,0 -2562,0 -2563,0 -2564,0 -2565,0 -2566,0 -2567,0 -2568,0 -2569,0 -2570,0 -2571,0 -2572,0 -2573,0 -2574,0 -2575,0 -2576,0 -2577,0 -2578,0 -2579,0 -2580,0 -2581,0 -2582,0 -2583,0 -2584,0 -2585,0 -2586,0 -2587,0 -2588,0 -2589,0 -2590,0 -2591,0 -2592,0 -2593,0 -2594,0 -2595,0 -2596,0 -2597,0 -2598,0 -2599,0 -2600,0 -2601,0 -2602,0 -2603,0 -2604,0 -2605,0 -2606,0 -2607,0 -2608,0 -2609,0 -2610,0 -2611,0 -2612,0 -2613,0 -2614,0 -2615,0 -2616,0 -2617,0 -2618,0 -2619,0 -2620,0 -2621,0 -2622,0 -2623,0 -2624,0 -2625,0 -2626,0 -2627,0 -2628,0 -2629,0 -2630,0 -2631,0 -2632,0 -2633,0 -2634,0 -2635,0 -2636,0 -2637,0 -2638,0 -2639,0 -2640,0 -2641,0 -2642,0 -2643,0 -2644,0 -2645,0 -2646,0 -2647,0 -2648,0 -2649,0 -2650,0 -2651,0 -2652,0 -2653,0 -2654,0 -2655,0 -2656,0 -2657,0 -2658,0 -2659,0 -2660,0 -2661,0 -2662,0 -2663,0 -2664,0 -2665,0 -2666,0 -2667,0 -2668,0 -2669,0 -2670,0 -2671,0 -2672,0 -2673,0 -2674,0 -2675,0 -2676,0 -2677,0 -2678,0 -2679,0 -2680,0 -2681,0 -2682,0 -2683,0 -2684,0 -2685,0 -2686,0 -2687,0 -2688,0 -2689,0 -2690,0 -2691,0 -2692,0 -2693,0 -2694,0 -2695,0 -2696,0 -2697,0 -2698,0 -2699,0 -2700,0 -2701,0 -2702,0 -2703,0 -2704,0 -2705,0 -2706,0 -2707,0 -2708,0 -2709,0 -2710,0 -2711,0 -2712,0 -2713,0 -2714,0 -2715,0 -2716,0 -2717,0 -2718,0 -2719,0 -2720,0 -2721,0 -2722,0 -2723,0 -2724,0 -2725,0 -2726,0 -2727,0 -2728,0 -2729,0 -2730,0 -2731,0 -2732,0 -2733,0 -2734,0 -2735,0 -2736,0 -2737,0 -2738,0 -2739,0 -2740,0 -2741,0 -2742,0 -2743,0 -2744,0 -2745,0 -2746,0 -2747,0 -2748,0 -2749,0 -2750,0 -2751,0 -2752,0 -2753,0 -2754,0 -2755,0 -2756,0 -2757,0 -2758,0 -2759,0 -2760,0 -2761,0 -2762,0 -2763,0 -2764,0 -2765,0 -2766,0 -2767,0 -2768,0 -2769,0 -2770,0 -2771,0 -2772,0 -2773,0 -2774,0 -2775,0 -2776,0 -2777,0 -2778,0 -2779,0 -2780,0 -2781,0 -2782,0 -2783,0 -2784,0 -2785,0 -2786,0 -2787,0 -2788,0 -2789,0 -2790,0 -2791,0 -2792,0 -2793,0 -2794,0 -2795,0 -2796,0 -2797,0 -2798,0 -2799,0 -2800,0 -2801,0 -2802,0 -2803,0 -2804,0 -2805,0 -2806,0 -2807,0 -2808,0 -2809,0 -2810,0 -2811,0 -2812,0 -2813,0 -2814,0 -2815,0 -2816,0 -2817,0 -2818,0 -2819,0 -2820,0 -2821,0 -2822,0 -2823,0 -2824,0 -2825,0 -2826,0 -2827,0 -2828,0 -2829,0 -2830,0 -2831,0 -2832,0 -2833,0 -2834,0 -2835,0 -2836,0 -2837,0 -2838,0 -2839,0 -2840,0 -2841,0 -2842,0 -2843,0 -2844,0 -2845,0 -2846,0 -2847,0 -2848,0 -2849,0 -2850,0 -2851,0 -2852,0 -2853,0 -2854,0 -2855,0 -2856,0 -2857,0 -2858,0 -2859,0 -2860,0 -2861,0 -2862,0 -2863,0 -2864,0 -2865,0 -2866,0 -2867,0 -2868,0 -2869,0 -2870,0 -2871,0 -2872,0 -2873,0 -2874,0 -2875,0 -2876,0 -2877,0 -2878,0 -2879,0 -2880,0 -2881,0 -2882,0 -2883,0 -2884,0 -2885,0 -2886,0 -2887,0 -2888,0 -2889,0 -2890,0 -2891,0 -2892,0 -2893,0 -2894,0 -2895,0 -2896,0 -2897,0 -2898,0 -2899,0 -2900,0 -2901,0 -2902,0 -2903,0 -2904,0 -2905,0 -2906,0 -2907,0 -2908,0 -2909,0 -2910,0 -2911,0 -2912,0 -2913,0 -2914,0 -2915,0 -2916,0 -2917,0 -2918,0 -2919,0 -2920,0 -2921,0 -2922,0 -2923,0 -2924,0 -2925,0 -2926,0 -2927,0 -2928,0 -2929,0 -2930,0 -2931,0 -2932,0 -2933,0 -2934,0 -2935,0 -2936,0 -2937,0 -2938,0 -2939,0 -2940,0 -2941,0 -2942,0 -2943,0 -2944,0 -2945,0 -2946,0 -2947,0 -2948,0 -2949,0 -2950,0 -2951,0 -2952,0 -2953,0 -2954,0 -2955,0 -2956,0 -2957,0 -2958,0 -2959,0 -2960,0 -2961,0 -2962,0 -2963,0 -2964,0 -2965,0 -2966,0 -2967,0 -2968,0 -2969,0 -2970,0 -2971,0 -2972,0 -2973,0 -2974,0 -2975,0 -2976,0 -2977,0 -2978,0 -2979,0 -2980,0 -2981,0 -2982,0 -2983,0 -2984,0 -2985,0 -2986,0 -2987,0 -2988,0 -2989,0 -2990,0 -2991,0 -2992,0 -2993,0 -2994,0 -2995,0 -2996,0 -2997,0 -2998,0 -2999,0 -3000,0 -3001,0 -3002,0 -3003,0 -3004,0 -3005,0 -3006,0 -3007,0 -3008,0 -3009,0 -3010,0 -3011,0 -3012,0 -3013,0 -3014,0 -3015,0 -3016,0 -3017,0 -3018,0 -3019,0 -3020,0 -3021,0 -3022,0 -3023,0 -3024,0 -3025,0 -3026,0 -3027,0 -3028,0 -3029,0 -3030,0 -3031,0 -3032,0 -3033,0 -3034,0 -3035,0 -3036,0 -3037,0 -3038,0 -3039,0 -3040,0 -3041,0 -3042,0 -3043,0 -3044,0 -3045,0 -3046,0 -3047,0 -3048,0 -3049,0 -3050,0 -3051,0 -3052,0 -3053,0 -3054,0 -3055,0 -3056,0 -3057,0 -3058,0 -3059,0 -3060,0 -3061,0 -3062,0 -3063,0 -3064,0 -3065,0 -3066,0 -3067,0 -3068,0 -3069,0 -3070,0 -3071,0 -3072,0 -3073,0 -3074,0 -3075,0 -3076,0 -3077,0 -3078,0 -3079,0 -3080,0 -3081,0 -3082,0 -3083,0 -3084,0 -3085,0 -3086,0 -3087,0 -3088,0 -3089,0 -3090,0 -3091,0 -3092,0 -3093,0 -3094,0 -3095,0 -3096,0 -3097,0 -3098,0 -3099,0 -3100,0 -3101,0 -3102,0 -3103,0 -3104,0 -3105,0 -3106,0 -3107,0 -3108,0 -3109,0 -3110,0 -3111,0 -3112,0 -3113,0 -3114,0 -3115,0 -3116,0 -3117,0 -3118,0 -3119,0 -3120,0 -3121,0 -3122,0 -3123,0 -3124,0 -3125,0 -3126,0 -3127,0 -3128,0 -3129,0 -3130,0 -3131,0 -3132,0 -3133,0 -3134,0 -3135,0 -3136,0 -3137,0 -3138,0 -3139,0 -3140,0 -3141,0 -3142,0 -3143,0 -3144,0 -3145,0 -3146,0 -3147,0 -3148,0 -3149,0 -3150,0 -3151,0 -3152,0 -3153,0 -3154,0 -3155,0 -3156,0 -3157,0 -3158,0 -3159,0 -3160,0 -3161,0 -3162,0 -3163,0 -3164,0 -3165,0 -3166,0 -3167,0 -3168,0 -3169,0 -3170,0 -3171,0 -3172,0 -3173,0 -3174,0 -3175,0 -3176,0 -3177,0 -3178,0 -3179,0 -3180,0 -3181,0 -3182,0 -3183,0 -3184,0 -3185,0 -3186,0 -3187,0 -3188,0 -3189,0 -3190,0 -3191,0 -3192,0 -3193,0 -3194,0 -3195,0 -3196,0 -3197,0 -3198,0 -3199,0 -3200,0 -3201,0 -3202,0 -3203,0 -3204,0 -3205,0 -3206,0 -3207,0 -3208,0 -3209,0 -3210,0 -3211,0 -3212,0 -3213,0 -3214,0 -3215,0 -3216,0 -3217,0 -3218,0 -3219,0 -3220,0 -3221,0 -3222,0 -3223,0 -3224,0 -3225,0 -3226,0 -3227,0 -3228,0 -3229,0 -3230,0 -3231,0 -3232,0 -3233,0 -3234,0 -3235,0 -3236,0 -3237,0 -3238,0 -3239,0 -3240,0 -3241,0 -3242,0 -3243,0 -3244,0 -3245,0 -3246,0 -3247,0 -3248,0 -3249,0 -3250,0 -3251,0 -3252,0 -3253,0 -3254,0 -3255,0 -3256,0 -3257,0 -3258,0 -3259,0 -3260,0 -3261,0 -3262,0 -3263,0 -3264,0 -3265,0 -3266,0 -3267,0 -3268,0 -3269,0 -3270,0 -3271,0 -3272,0 -3273,0 -3274,0 -3275,0 -3276,0 -3277,0 -3278,0 -3279,0 -3280,0 -3281,0 -3282,0 -3283,0 -3284,0 -3285,0 -3286,0 -3287,0 -3288,0 -3289,0 -3290,0 -3291,0 -3292,0 -3293,0 -3294,0 -3295,0 -3296,0 -3297,0 -3298,0 -3299,0 -3300,0 -3301,0 -3302,0 -3303,0 -3304,0 -3305,0 -3306,0 -3307,0 -3308,0 -3309,0 -3310,0 -3311,0 -3312,0 -3313,0 -3314,0 -3315,0 -3316,0 -3317,0 -3318,0 -3319,0 -3320,0 -3321,0 -3322,0 -3323,0 -3324,0 -3325,0 -3326,0 -3327,0 -3328,0 -3329,0 -3330,0 -3331,0 -3332,0 -3333,0 -3334,0 -3335,0 -3336,0 -3337,0 -3338,0 -3339,0 -3340,0 -3341,0 -3342,0 -3343,0 -3344,0 -3345,0 -3346,0 -3347,0 -3348,0 -3349,0 -3350,0 -3351,0 -3352,0 -3353,0 -3354,0 -3355,0 -3356,0 -3357,0 -3358,0 -3359,0 -3360,0 -3361,0 -3362,0 -3363,0 -3364,0 -3365,0 -3366,0 -3367,0 -3368,0 -3369,0 -3370,0 -3371,0 -3372,0 -3373,0 -3374,0 -3375,0 -3376,0 -3377,0 -3378,0 -3379,0 -3380,0 -3381,0 -3382,0 -3383,0 -3384,0 -3385,0 -3386,0 -3387,0 -3388,0 -3389,0 -3390,0 -3391,0 -3392,0 -3393,0 -3394,0 -3395,0 -3396,0 -3397,0 -3398,0 -3399,0 -3400,0 -3401,0 -3402,0 -3403,0 -3404,0 -3405,0 -3406,0 -3407,0 -3408,0 -3409,0 -3410,0 -3411,0 -3412,0 -3413,0 -3414,0 -3415,0 -3416,0 -3417,0 -3418,0 -3419,0 -3420,0 -3421,0 -3422,0 -3423,0 -3424,0 -3425,0 -3426,0 -3427,0 -3428,0 -3429,0 -3430,0 -3431,0 -3432,0 -3433,0 -3434,0 -3435,0 -3436,0 -3437,0 -3438,0 -3439,0 -3440,0 -3441,0 -3442,0 -3443,0 -3444,0 -3445,0 -3446,0 -3447,0 -3448,0 -3449,0 -3450,0 -3451,0 -3452,0 -3453,0 -3454,0 -3455,0 -3456,0 -3457,0 -3458,0 -3459,0 -3460,0 -3461,0 -3462,0 -3463,0 -3464,0 -3465,0 -3466,0 -3467,0 -3468,0 -3469,0 -3470,0 -3471,0 -3472,0 -3473,0 -3474,0 -3475,0 -3476,0 -3477,0 -3478,0 -3479,0 -3480,0 -3481,0 -3482,0 -3483,0 -3484,0 -3485,0 -3486,0 -3487,0 -3488,0 -3489,0 -3490,0 -3491,0 -3492,0 -3493,0 -3494,0 -3495,0 -3496,0 -3497,0 -3498,0 -3499,0 -3500,0 -3501,0 -3502,0 -3503,0 -3504,0 -3505,0 -3506,0 -3507,0 -3508,0 -3509,0 -3510,0 -3511,0 -3512,0 -3513,0 -3514,0 -3515,0 -3516,0 -3517,0 -3518,0 -3519,0 -3520,0 -3521,0 -3522,0 -3523,0 -3524,0 -3525,0 -3526,0 -3527,0 -3528,0 -3529,0 -3530,0 -3531,0 -3532,0 -3533,0 -3534,0 -3535,0 -3536,0 -3537,0 -3538,0 -3539,0 -3540,0 -3541,0 -3542,0 -3543,0 -3544,0 -3545,0 -3546,0 -3547,0 -3548,0 -3549,0 -3550,0 -3551,0 -3552,0 -3553,0 -3554,0 -3555,0 -3556,0 -3557,0 -3558,0 -3559,0 -3560,0 -3561,0 -3562,0 -3563,0 -3564,0 -3565,0 -3566,0 -3567,0 -3568,0 -3569,0 -3570,0 -3571,0 -3572,0 -3573,0 -3574,0 -3575,0 -3576,0 -3577,0 -3578,0 -3579,0 -3580,0 -3581,0 -3582,0 -3583,0 -3584,0 -3585,0 -3586,0 -3587,0 -3588,0 -3589,0 -3590,0 -3591,0 -3592,0 -3593,0 -3594,0 -3595,0 -3596,0 -3597,0 -3598,0 -3599,0 -3600,0 -3601,0 -3602,0 -3603,0 -3604,0 -3605,0 -3606,0 -3607,0 -3608,0 -3609,0 -3610,0 -3611,0 -3612,0 -3613,0 -3614,0 -3615,0 -3616,0 -3617,0 -3618,0 -3619,0 -3620,0 -3621,0 -3622,0 -3623,0 -3624,0 -3625,0 -3626,0 -3627,0 -3628,0 -3629,0 -3630,0 -3631,0 -3632,0 -3633,0 -3634,0 -3635,0 -3636,0 -3637,0 -3638,0 -3639,0 -3640,0 -3641,0 -3642,0 -3643,0 -3644,0 -3645,0 -3646,0 -3647,0 -3648,0 -3649,0 -3650,0 -3651,0 -3652,0 -3653,0 -3654,0 -3655,0 -3656,0 -3657,0 -3658,0 -3659,0 -3660,0 -3661,0 -3662,0 -3663,0 -3664,0 -3665,0 -3666,0 -3667,0 -3668,0 -3669,0 -3670,0 -3671,0 -3672,0 -3673,0 -3674,0 -3675,0 -3676,0 -3677,0 -3678,0 -3679,0 -3680,0 -3681,0 -3682,0 -3683,0 -3684,0 -3685,0 -3686,0 -3687,0 -3688,0 -3689,0 -3690,0 -3691,0 -3692,0 -3693,0 -3694,0 -3695,0 -3696,0 -3697,0 -3698,0 -3699,0 -3700,0 -3701,0 -3702,0 -3703,0 -3704,0 -3705,0 -3706,0 -3707,0 -3708,0 -3709,0 -3710,0 -3711,0 -3712,0 -3713,0 -3714,0 -3715,0 -3716,0 -3717,0 -3718,0 -3719,0 -3720,0 -3721,0 -3722,0 -3723,0 -3724,0 -3725,0 -3726,0 -3727,0 -3728,0 -3729,0 -3730,0 -3731,0 -3732,0 -3733,0 -3734,0 -3735,0 -3736,0 -3737,0 -3738,0 -3739,0 -3740,0 -3741,0 -3742,0 -3743,0 -3744,0 -3745,0 -3746,0 -3747,0 -3748,0 -3749,0 -3750,0 -3751,0 -3752,0 -3753,0 -3754,0 -3755,0 -3756,0 -3757,0 -3758,0 -3759,0 -3760,0 -3761,0 -3762,0 -3763,0 -3764,0 -3765,0 -3766,0 -3767,0 -3768,0 -3769,0 -3770,0 -3771,0 -3772,0 -3773,0 -3774,0 -3775,0 -3776,0 -3777,0 -3778,0 -3779,0 -3780,0 -3781,0 -3782,0 -3783,0 -3784,0 -3785,0 -3786,0 -3787,0 -3788,0 -3789,0 -3790,0 -3791,0 -3792,0 -3793,0 -3794,0 -3795,0 -3796,0 -3797,0 -3798,0 -3799,0 -3800,0 -3801,0 -3802,0 -3803,0 -3804,0 -3805,0 -3806,0 -3807,0 -3808,0 -3809,0 -3810,0 -3811,0 -3812,0 -3813,0 -3814,0 -3815,0 -3816,0 -3817,0 -3818,0 -3819,0 -3820,0 -3821,0 -3822,0 -3823,0 -3824,0 -3825,0 -3826,0 -3827,0 -3828,0 -3829,0 -3830,0 -3831,0 -3832,0 -3833,0 -3834,0 -3835,0 -3836,0 -3837,0 -3838,0 -3839,0 -3840,0 -3841,0 -3842,0 -3843,0 -3844,0 -3845,0 -3846,0 -3847,0 -3848,0 -3849,0 -3850,0 -3851,0 -3852,0 -3853,0 -3854,0 -3855,0 -3856,0 -3857,0 -3858,0 -3859,0 -3860,0 -3861,0 -3862,0 -3863,0 -3864,0 -3865,0 -3866,0 -3867,0 -3868,0 -3869,0 -3870,0 -3871,0 -3872,0 -3873,0 -3874,0 -3875,0 -3876,0 -3877,0 -3878,0 -3879,0 -3880,0 -3881,0 -3882,0 -3883,0 -3884,0 -3885,0 -3886,0 -3887,0 -3888,0 -3889,0 -3890,0 -3891,0 -3892,0 -3893,0 -3894,0 -3895,0 -3896,0 -3897,0 -3898,0 -3899,0 -3900,0 -3901,0 -3902,0 -3903,0 -3904,0 -3905,0 -3906,0 -3907,0 -3908,0 -3909,0 -3910,0 -3911,0 -3912,0 -3913,0 -3914,0 -3915,0 -3916,0 -3917,0 -3918,0 -3919,0 -3920,0 -3921,0 -3922,0 -3923,0 -3924,0 -3925,0 -3926,0 -3927,0 -3928,0 -3929,0 -3930,0 -3931,0 -3932,0 -3933,0 -3934,0 -3935,0 -3936,0 -3937,0 -3938,0 -3939,0 -3940,0 -3941,0 -3942,0 -3943,0 -3944,0 -3945,0 -3946,0 -3947,0 -3948,0 -3949,0 -3950,0 -3951,0 -3952,0 -3953,0 -3954,0 -3955,0 -3956,0 -3957,0 -3958,0 -3959,0 -3960,0 -3961,0 -3962,0 -3963,0 -3964,0 -3965,0 -3966,0 -3967,0 -3968,0 -3969,0 -3970,0 -3971,0 -3972,0 -3973,0 -3974,0 -3975,0 -3976,0 -3977,0 -3978,0 -3979,0 -3980,0 -3981,0 -3982,0 -3983,0 -3984,0 -3985,0 -3986,0 -3987,0 -3988,0 -3989,0 -3990,0 -3991,0 -3992,0 -3993,0 -3994,0 -3995,0 -3996,0 -3997,0 -3998,0 -3999,0 -4000,0 -4001,0 -4002,0 -4003,0 -4004,0 -4005,0 -4006,0 -4007,0 -4008,0 -4009,0 -4010,0 -4011,0 -4012,0 -4013,0 -4014,0 -4015,0 -4016,0 -4017,0 -4018,0 -4019,0 -4020,0 -4021,0 -4022,0 -4023,0 -4024,0 -4025,0 -4026,0 -4027,0 -4028,0 -4029,0 -4030,0 -4031,0 -4032,0 -4033,0 -4034,0 -4035,0 -4036,0 -4037,0 -4038,0 -4039,0 -4040,0 -4041,0 -4042,0 -4043,0 -4044,0 -4045,0 -4046,0 -4047,0 -4048,0 -4049,0 -4050,0 -4051,0 -4052,0 -4053,0 -4054,0 -4055,0 -4056,0 -4057,0 -4058,0 -4059,0 -4060,0 -4061,0 -4062,0 -4063,0 -4064,0 -4065,0 -4066,0 -4067,0 -4068,0 -4069,0 -4070,0 -4071,0 -4072,0 -4073,0 -4074,0 -4075,0 -4076,0 -4077,0 -4078,0 -4079,0 -4080,0 -4081,0 -4082,0 -4083,0 -4084,0 -4085,0 -4086,0 -4087,0 -4088,0 -4089,0 -4090,0 -4091,0 -4092,0 -4093,0 -4094,0 -4095,0 -4096,0 -4097,0 -4098,0 -4099,0 -4100,0 -4101,0 -4102,0 -4103,0 -4104,0 -4105,0 -4106,0 -4107,0 -4108,0 -4109,0 -4110,0 -4111,0 -4112,0 -4113,0 -4114,0 -4115,0 -4116,0 -4117,0 -4118,0 -4119,0 -4120,0 -4121,0 -4122,0 -4123,0 -4124,0 -4125,0 -4126,0 -4127,0 -4128,0 -4129,0 -4130,0 -4131,0 -4132,0 -4133,0 -4134,0 -4135,0 -4136,0 -4137,0 -4138,0 -4139,0 -4140,0 -4141,0 -4142,0 -4143,0 -4144,0 -4145,0 -4146,0 -4147,0 -4148,0 -4149,0 -4150,0 -4151,0 -4152,0 -4153,0 -4154,0 -4155,0 -4156,0 -4157,0 -4158,0 -4159,0 -4160,0 -4161,0 -4162,0 -4163,0 -4164,0 -4165,0 -4166,0 -4167,0 -4168,0 -4169,0 -4170,0 -4171,0 -4172,0 -4173,0 -4174,0 -4175,0 -4176,0 -4177,0 -4178,0 -4179,0 -4180,0 -4181,0 -4182,0 -4183,0 -4184,0 -4185,0 -4186,0 -4187,0 -4188,0 -4189,0 -4190,0 -4191,0 -4192,0 -4193,0 -4194,0 -4195,0 -4196,0 -4197,0 -4198,0 -4199,0 -4200,0 -4201,0 -4202,0 -4203,0 -4204,0 -4205,0 -4206,0 -4207,0 -4208,0 -4209,0 -4210,0 -4211,0 -4212,0 -4213,0 -4214,0 -4215,0 -4216,0 -4217,0 -4218,0 -4219,0 -4220,0 -4221,0 -4222,0 -4223,0 -4224,0 -4225,0 -4226,0 -4227,0 -4228,0 -4229,0 -4230,0 -4231,0 -4232,0 -4233,0 -4234,0 -4235,0 -4236,0 -4237,0 -4238,0 -4239,0 -4240,0 -4241,0 -4242,0 -4243,0 -4244,0 -4245,0 -4246,0 -4247,0 -4248,0 -4249,0 -4250,0 -4251,0 -4252,0 -4253,0 -4254,0 -4255,0 -4256,0 -4257,0 -4258,0 -4259,0 -4260,0 -4261,0 -4262,0 -4263,0 -4264,0 -4265,0 -4266,0 -4267,0 -4268,0 -4269,0 -4270,0 -4271,0 -4272,0 -4273,0 -4274,0 -4275,0 -4276,0 -4277,0 -4278,0 -4279,0 -4280,0 -4281,0 -4282,0 -4283,0 -4284,0 -4285,0 -4286,0 -4287,0 -4288,0 -4289,0 -4290,0 -4291,0 -4292,0 -4293,0 -4294,0 -4295,0 -4296,0 -4297,0 -4298,0 -4299,0 -4300,0 -4301,0 -4302,0 -4303,0 -4304,0 -4305,0 -4306,0 -4307,0 -4308,0 -4309,0 -4310,0 -4311,0 -4312,0 -4313,0 -4314,0 -4315,0 -4316,0 -4317,0 -4318,0 -4319,0 -4320,0 -4321,0 -4322,0 -4323,0 -4324,0 -4325,0 -4326,0 -4327,0 -4328,0 -4329,0 -4330,0 -4331,0 -4332,0 -4333,0 -4334,0 -4335,0 -4336,0 -4337,0 -4338,0 -4339,0 -4340,0 -4341,0 -4342,0 -4343,0 -4344,0 -4345,0 -4346,0 -4347,0 -4348,0 -4349,0 -4350,0 -4351,0 -4352,0 -4353,0 -4354,0 -4355,0 -4356,0 -4357,0 -4358,0 -4359,0 -4360,0 -4361,0 -4362,0 -4363,0 -4364,0 -4365,0 -4366,0 -4367,0 -4368,0 -4369,0 -4370,0 -4371,0 -4372,0 -4373,0 -4374,0 -4375,0 -4376,0 -4377,0 -4378,0 -4379,0 -4380,0 -4381,0 -4382,0 -4383,0 -4384,0 -4385,0 -4386,0 -4387,0 -4388,0 -4389,0 -4390,0 -4391,0 -4392,0 -4393,0 -4394,0 -4395,0 -4396,0 -4397,0 -4398,0 -4399,0 -4400,0 -4401,0 -4402,0 -4403,0 -4404,0 -4405,0 -4406,0 -4407,0 -4408,0 -4409,0 -4410,0 -4411,0 -4412,0 -4413,0 -4414,0 -4415,0 -4416,0 -4417,0 -4418,0 -4419,0 -4420,0 -4421,0 -4422,0 -4423,0 -4424,0 -4425,0 -4426,0 -4427,0 -4428,0 -4429,0 -4430,0 -4431,0 -4432,0 -4433,0 -4434,0 -4435,0 -4436,0 -4437,0 -4438,0 -4439,0 -4440,0 -4441,0 -4442,0 -4443,0 -4444,0 -4445,0 -4446,0 -4447,0 -4448,0 -4449,0 -4450,0 -4451,0 -4452,0 -4453,0 -4454,0 -4455,0 -4456,0 -4457,0 -4458,0 -4459,0 -4460,0 -4461,0 -4462,0 -4463,0 -4464,0 -4465,0 -4466,0 -4467,0 -4468,0 -4469,0 -4470,0 -4471,0 -4472,0 -4473,0 -4474,0 -4475,0 -4476,0 -4477,0 -4478,0 -4479,0 -4480,0 -4481,0 -4482,0 -4483,0 -4484,0 -4485,0 -4486,0 -4487,0 -4488,0 -4489,0 -4490,0 -4491,0 -4492,0 -4493,0 -4494,0 -4495,0 -4496,0 -4497,0 -4498,0 -4499,0 -4500,0 -4501,0 -4502,0 -4503,0 -4504,0 -4505,0 -4506,0 -4507,0 -4508,0 -4509,0 -4510,0 -4511,0 -4512,0 -4513,0 -4514,0 -4515,0 -4516,0 -4517,0 -4518,0 -4519,0 -4520,0 -4521,0 -4522,0 -4523,0 -4524,0 -4525,0 -4526,0 -4527,0 -4528,0 -4529,0 -4530,0 -4531,0 -4532,0 -4533,0 -4534,0 -4535,0 -4536,0 -4537,0 -4538,0 -4539,0 -4540,0 -4541,0 -4542,0 -4543,0 -4544,0 -4545,0 -4546,0 -4547,0 -4548,0 -4549,0 -4550,0 -4551,0 -4552,0 -4553,0 -4554,0 -4555,0 -4556,0 -4557,0 -4558,0 -4559,0 -4560,0 -4561,0 -4562,0 -4563,0 -4564,0 -4565,0 -4566,0 -4567,0 -4568,0 -4569,0 -4570,0 -4571,0 -4572,0 -4573,0 -4574,0 -4575,0 -4576,0 -4577,0 -4578,0 -4579,0 -4580,0 -4581,0 -4582,0 -4583,0 -4584,0 -4585,0 -4586,0 -4587,0 -4588,0 -4589,0 -4590,0 -4591,0 -4592,0 -4593,0 -4594,0 -4595,0 -4596,0 -4597,0 -4598,0 -4599,0 -4600,0 -4601,0 -4602,0 -4603,0 -4604,0 -4605,0 -4606,0 -4607,0 -4608,0 -4609,0 -4610,0 -4611,0 -4612,0 -4613,0 -4614,0 -4615,0 -4616,0 -4617,0 -4618,0 -4619,0 -4620,0 -4621,0 -4622,0 -4623,0 -4624,0 -4625,0 -4626,0 -4627,0 -4628,0 -4629,0 -4630,0 -4631,0 -4632,0 -4633,0 -4634,0 -4635,0 -4636,0 -4637,0 -4638,0 -4639,0 -4640,0 -4641,0 -4642,0 -4643,0 -4644,0 -4645,0 -4646,0 -4647,0 -4648,0 -4649,0 -4650,0 -4651,0 -4652,0 -4653,0 -4654,0 -4655,0 -4656,0 -4657,0 -4658,0 -4659,0 -4660,0 -4661,0 -4662,0 -4663,0 -4664,0 -4665,0 -4666,0 -4667,0 -4668,0 -4669,0 -4670,0 -4671,0 -4672,0 -4673,0 -4674,0 -4675,0 -4676,0 -4677,0 -4678,0 -4679,0 -4680,0 -4681,0 -4682,0 -4683,0 -4684,0 -4685,0 -4686,0 -4687,0 -4688,0 -4689,0 -4690,0 -4691,0 -4692,0 -4693,0 -4694,0 -4695,0 -4696,0 -4697,0 -4698,0 -4699,0 -4700,0 -4701,0 -4702,0 -4703,0 -4704,0 -4705,0 -4706,0 -4707,0 -4708,0 -4709,0 -4710,0 -4711,0 -4712,0 -4713,0 -4714,0 -4715,0 -4716,0 -4717,0 -4718,0 -4719,0 -4720,0 -4721,0 -4722,0 -4723,0 -4724,0 -4725,0 -4726,0 -4727,0 -4728,0 -4729,0 -4730,0 -4731,0 -4732,0 -4733,0 -4734,0 -4735,0 -4736,0 -4737,0 -4738,0 -4739,0 -4740,0 -4741,0 -4742,0 -4743,0 -4744,0 -4745,0 -4746,0 -4747,0 -4748,0 -4749,0 -4750,0 -4751,0 -4752,0 -4753,0 -4754,0 -4755,0 -4756,0 -4757,0 -4758,0 -4759,0 -4760,0 -4761,0 -4762,0 -4763,0 -4764,0 -4765,0 -4766,0 -4767,0 -4768,0 -4769,0 -4770,0 -4771,0 -4772,0 -4773,0 -4774,0 -4775,0 -4776,0 -4777,0 -4778,0 -4779,0 -4780,0 -4781,0 -4782,0 -4783,0 -4784,0 -4785,0 -4786,0 -4787,0 -4788,0 -4789,0 -4790,0 -4791,0 -4792,0 -4793,0 -4794,0 -4795,0 -4796,0 -4797,0 -4798,0 -4799,0 -4800,0 -4801,0 -4802,0 -4803,0 -4804,0 -4805,0 -4806,0 -4807,0 -4808,0 -4809,0 -4810,0 -4811,0 -4812,0 -4813,0 -4814,0 -4815,0 -4816,0 -4817,0 -4818,0 -4819,0 -4820,0 -4821,0 -4822,0 -4823,0 -4824,0 -4825,0 -4826,0 -4827,0 -4828,0 -4829,0 -4830,0 -4831,0 -4832,0 -4833,0 -4834,0 -4835,0 -4836,0 -4837,0 -4838,0 -4839,0 -4840,0 -4841,0 -4842,0 -4843,0 -4844,0 -4845,0 -4846,0 -4847,0 -4848,0 -4849,0 -4850,0 -4851,0 -4852,0 -4853,0 -4854,0 -4855,0 -4856,0 -4857,0 -4858,0 -4859,0 -4860,0 -4861,0 -4862,0 -4863,0 -4864,0 -4865,0 -4866,0 -4867,0 -4868,0 -4869,0 -4870,0 -4871,0 -4872,0 -4873,0 -4874,0 -4875,0 -4876,0 -4877,0 -4878,0 -4879,0 -4880,0 -4881,0 -4882,0 -4883,0 -4884,0 -4885,0 -4886,0 -4887,0 -4888,0 -4889,0 -4890,0 -4891,0 -4892,0 -4893,0 -4894,0 -4895,0 -4896,0 -4897,0 -4898,0 -4899,0 -4900,0 -4901,0 -4902,0 -4903,0 -4904,0 -4905,0 -4906,0 -4907,0 -4908,0 -4909,0 -4910,0 -4911,0 -4912,0 -4913,0 -4914,0 -4915,0 -4916,0 -4917,0 -4918,0 -4919,0 -4920,0 -4921,0 -4922,0 -4923,0 -4924,0 -4925,0 -4926,0 -4927,0 -4928,0 -4929,0 -4930,0 -4931,0 -4932,0 -4933,0 -4934,0 -4935,0 -4936,0 -4937,0 -4938,0 -4939,0 -4940,0 -4941,0 -4942,0 -4943,0 -4944,0 -4945,0 -4946,0 -4947,0 -4948,0 -4949,0 -4950,0 -4951,0 -4952,0 -4953,0 -4954,0 -4955,0 -4956,0 -4957,0 -4958,0 -4959,0 -4960,0 -4961,0 -4962,0 -4963,0 -4964,0 -4965,0 -4966,0 -4967,0 -4968,0 -4969,0 -4970,0 -4971,0 -4972,0 -4973,0 -4974,0 -4975,0 -4976,0 -4977,0 -4978,0 -4979,0 -4980,0 -4981,0 -4982,0 -4983,0 -4984,0 -4985,0 -4986,0 -4987,0 -4988,0 -4989,0 -4990,0 -4991,0 -4992,0 -4993,0 -4994,0 -4995,0 -4996,0 -4997,0 -4998,0 -4999,0 -5000,0 -5001,0 -5002,0 -5003,0 -5004,0 -5005,0 -5006,0 -5007,0 -5008,0 -5009,0 -5010,0 -5011,0 -5012,0 -5013,0 -5014,0 -5015,0 -5016,0 -5017,0 -5018,0 -5019,0 -5020,0 -5021,0 -5022,0 -5023,0 -5024,0 -5025,0 -5026,0 -5027,0 -5028,0 -5029,0 -5030,0 -5031,0 -5032,0 -5033,0 -5034,0 -5035,0 -5036,0 -5037,0 -5038,0 -5039,0 -5040,0 -5041,0 -5042,0 -5043,0 -5044,0 -5045,0 -5046,0 -5047,0 -5048,0 -5049,0 -5050,0 -5051,0 -5052,0 -5053,0 -5054,0 -5055,0 -5056,0 -5057,0 -5058,0 -5059,0 -5060,0 -5061,0 -5062,0 -5063,0 -5064,0 -5065,0 -5066,0 -5067,0 -5068,0 -5069,0 -5070,0 -5071,0 -5072,0 -5073,0 -5074,0 -5075,0 -5076,0 -5077,0 -5078,0 -5079,0 -5080,0 -5081,0 -5082,0 -5083,0 -5084,0 -5085,0 -5086,0 -5087,0 -5088,0 -5089,0 -5090,0 -5091,0 -5092,0 -5093,0 -5094,0 -5095,0 -5096,0 -5097,0 -5098,0 -5099,0 -5100,0 -5101,0 -5102,0 -5103,0 -5104,0 -5105,0 -5106,0 -5107,0 -5108,0 -5109,0 -5110,0 -5111,0 -5112,0 -5113,0 -5114,0 -5115,0 -5116,0 -5117,0 -5118,0 -5119,0 -5120,0 -5121,0 -5122,0 -5123,0 -5124,0 -5125,0 -5126,0 -5127,0 -5128,0 -5129,0 -5130,0 -5131,0 -5132,0 -5133,0 -5134,0 -5135,0 -5136,0 -5137,0 -5138,0 -5139,0 -5140,0 -5141,0 -5142,0 -5143,0 -5144,0 -5145,0 -5146,0 -5147,0 -5148,0 -5149,0 -5150,0 -5151,0 -5152,0 -5153,0 -5154,0 -5155,0 -5156,0 -5157,0 -5158,0 -5159,0 -5160,0 -5161,0 -5162,0 -5163,0 -5164,0 -5165,0 -5166,0 -5167,0 -5168,0 -5169,0 -5170,0 -5171,0 -5172,0 -5173,0 -5174,0 -5175,0 -5176,0 -5177,0 -5178,0 -5179,0 -5180,0 -5181,0 -5182,0 -5183,0 -5184,0 -5185,0 -5186,0 -5187,0 -5188,0 -5189,0 -5190,0 -5191,0 -5192,0 -5193,0 -5194,0 -5195,0 -5196,0 -5197,0 -5198,0 -5199,0 -5200,0 -5201,0 -5202,0 -5203,0 -5204,0 -5205,0 -5206,0 -5207,0 -5208,0 -5209,0 -5210,0 -5211,0 -5212,0 -5213,0 -5214,0 -5215,0 -5216,0 -5217,0 -5218,0 -5219,0 -5220,0 -5221,0 -5222,0 -5223,0 -5224,0 -5225,0 -5226,0 -5227,0 -5228,0 -5229,0 -5230,0 -5231,0 -5232,0 -5233,0 -5234,0 -5235,0 -5236,0 -5237,0 -5238,0 -5239,0 -5240,0 -5241,0 -5242,0 -5243,0 -5244,0 -5245,0 -5246,0 -5247,0 -5248,0 -5249,0 -5250,0 -5251,0 -5252,0 -5253,0 -5254,0 -5255,0 -5256,0 -5257,0 -5258,0 -5259,0 -5260,0 -5261,0 -5262,0 -5263,0 -5264,0 -5265,0 -5266,0 -5267,0 -5268,0 -5269,0 -5270,0 -5271,0 -5272,0 -5273,0 -5274,0 -5275,0 -5276,0 -5277,0 -5278,0 -5279,0 -5280,0 -5281,0 -5282,0 -5283,0 -5284,0 -5285,0 -5286,0 -5287,0 -5288,0 -5289,0 -5290,0 -5291,0 -5292,0 -5293,0 -5294,0 -5295,0 -5296,0 -5297,0 -5298,0 -5299,0 -5300,0 -5301,0 -5302,0 -5303,0 -5304,0 -5305,0 -5306,0 -5307,0 -5308,0 -5309,0 -5310,0 -5311,0 -5312,0 -5313,0 -5314,0 -5315,0 -5316,0 -5317,0 -5318,0 -5319,0 -5320,0 -5321,0 -5322,0 -5323,0 -5324,0 -5325,0 -5326,0 -5327,0 -5328,0 -5329,0 -5330,0 -5331,0 -5332,0 -5333,0 -5334,0 -5335,0 -5336,0 -5337,0 -5338,0 -5339,0 -5340,0 -5341,0 -5342,0 -5343,0 -5344,0 -5345,0 -5346,0 -5347,0 -5348,0 -5349,0 -5350,0 -5351,0 -5352,0 -5353,0 -5354,0 -5355,0 -5356,0 -5357,0 -5358,0 -5359,0 -5360,0 -5361,0 -5362,0 -5363,0 -5364,0 -5365,0 -5366,0 -5367,0 -5368,0 -5369,0 -5370,0 -5371,0 -5372,0 -5373,0 -5374,0 -5375,0 -5376,0 -5377,0 -5378,0 -5379,0 -5380,0 -5381,0 -5382,0 -5383,0 -5384,0 -5385,0 -5386,0 -5387,0 -5388,0 -5389,0 -5390,0 -5391,0 -5392,0 -5393,0 -5394,0 -5395,0 -5396,0 -5397,0 -5398,0 -5399,0 -5400,0 -5401,0 -5402,0 -5403,0 -5404,0 -5405,0 -5406,0 -5407,0 -5408,0 -5409,0 -5410,0 -5411,0 -5412,0 -5413,0 -5414,0 -5415,0 -5416,0 -5417,0 -5418,0 -5419,0 -5420,0 -5421,0 -5422,0 -5423,0 -5424,0 -5425,0 -5426,0 -5427,0 -5428,0 -5429,0 -5430,0 -5431,0 -5432,0 -5433,0 -5434,0 -5435,0 -5436,0 -5437,0 -5438,0 -5439,0 -5440,0 -5441,0 -5442,0 -5443,0 -5444,0 -5445,0 -5446,0 -5447,0 -5448,0 -5449,0 -5450,0 -5451,0 -5452,0 -5453,0 -5454,0 -5455,0 -5456,0 -5457,0 -5458,0 -5459,0 -5460,0 -5461,0 -5462,0 -5463,0 -5464,0 -5465,0 -5466,0 -5467,0 -5468,0 -5469,0 -5470,0 -5471,0 -5472,0 -5473,0 -5474,0 -5475,0 -5476,0 -5477,0 -5478,0 -5479,0 -5480,0 -5481,0 -5482,0 -5483,0 -5484,0 -5485,0 -5486,0 -5487,0 -5488,0 -5489,0 -5490,0 -5491,0 -5492,0 -5493,0 -5494,0 -5495,0 -5496,0 -5497,0 -5498,0 -5499,0 -5500,0 -5501,0 -5502,0 -5503,0 -5504,0 -5505,0 -5506,0 -5507,0 -5508,0 -5509,0 -5510,0 -5511,0 -5512,0 -5513,0 -5514,0 -5515,0 -5516,0 -5517,0 -5518,0 -5519,0 -5520,0 -5521,0 -5522,0 -5523,0 -5524,0 -5525,0 -5526,0 -5527,0 -5528,0 -5529,0 -5530,0 -5531,0 -5532,0 -5533,0 -5534,0 -5535,0 -5536,0 -5537,0 -5538,0 -5539,0 -5540,0 -5541,0 -5542,0 -5543,0 -5544,0 -5545,0 -5546,0 -5547,0 -5548,0 -5549,0 -5550,0 -5551,0 -5552,0 -5553,0 -5554,0 -5555,0 -5556,0 -5557,0 -5558,0 -5559,0 -5560,0 -5561,0 -5562,0 -5563,0 -5564,0 -5565,0 -5566,0 -5567,0 -5568,0 -5569,0 -5570,0 -5571,0 -5572,0 -5573,0 -5574,0 -5575,0 -5576,0 -5577,0 -5578,0 -5579,0 -5580,0 -5581,0 -5582,0 -5583,0 -5584,0 -5585,0 -5586,0 -5587,0 -5588,0 -5589,0 -5590,0 -5591,0 -5592,0 -5593,0 -5594,0 -5595,0 -5596,0 -5597,0 -5598,0 -5599,0 -5600,0 -5601,0 -5602,0 -5603,0 -5604,0 -5605,0 -5606,0 -5607,0 -5608,0 -5609,0 -5610,0 -5611,0 -5612,0 -5613,0 -5614,0 -5615,0 -5616,0 -5617,0 -5618,0 -5619,0 -5620,0 -5621,0 -5622,0 -5623,0 -5624,0 -5625,0 -5626,0 -5627,0 -5628,0 -5629,0 -5630,0 -5631,0 -5632,0 -5633,0 -5634,0 -5635,0 -5636,0 -5637,0 -5638,0 -5639,0 -5640,0 -5641,0 -5642,0 -5643,0 -5644,0 -5645,0 -5646,0 -5647,0 -5648,0 -5649,0 -5650,0 -5651,0 -5652,0 -5653,0 -5654,0 -5655,0 -5656,0 -5657,0 -5658,0 -5659,0 -5660,0 -5661,0 -5662,0 -5663,0 -5664,0 -5665,0 -5666,0 -5667,0 -5668,0 -5669,0 -5670,0 -5671,0 -5672,0 -5673,0 -5674,0 -5675,0 -5676,0 -5677,0 -5678,0 -5679,0 -5680,0 -5681,0 -5682,0 -5683,0 -5684,0 -5685,0 -5686,0 -5687,0 -5688,0 -5689,0 -5690,0 -5691,0 -5692,0 -5693,0 -5694,0 -5695,0 -5696,0 -5697,0 -5698,0 -5699,0 -5700,0 -5701,0 -5702,0 -5703,0 -5704,0 -5705,0 -5706,0 -5707,0 -5708,0 -5709,0 -5710,0 -5711,0 -5712,0 -5713,0 -5714,0 -5715,0 -5716,0 -5717,0 -5718,0 -5719,0 -5720,0 -5721,0 -5722,0 -5723,0 -5724,0 -5725,0 -5726,0 -5727,0 -5728,0 -5729,0 -5730,0 -5731,0 -5732,0 -5733,0 -5734,0 -5735,0 -5736,0 -5737,0 -5738,0 -5739,0 -5740,0 -5741,0 -5742,0 -5743,0 -5744,0 -5745,0 -5746,0 -5747,0 -5748,0 -5749,0 -5750,0 -5751,0 -5752,0 -5753,0 -5754,0 -5755,0 -5756,0 -5757,0 -5758,0 -5759,0 -5760,0 -5761,0 -5762,0 -5763,0 -5764,0 -5765,0 -5766,0 -5767,0 -5768,0 -5769,0 -5770,0 -5771,0 -5772,0 -5773,0 -5774,0 -5775,0 -5776,0 -5777,0 -5778,0 -5779,0 -5780,0 -5781,0 -5782,0 -5783,0 -5784,0 -5785,0 -5786,0 -5787,0 -5788,0 -5789,0 -5790,0 -5791,0 -5792,0 -5793,0 -5794,0 -5795,0 -5796,0 -5797,0 -5798,0 -5799,0 -5800,0 -5801,0 -5802,0 -5803,0 -5804,0 -5805,0 -5806,0 -5807,0 -5808,0 -5809,0 -5810,0 -5811,0 -5812,0 -5813,0 -5814,0 -5815,0 -5816,0 -5817,0 -5818,0 -5819,0 -5820,0 -5821,0 -5822,0 -5823,0 -5824,0 -5825,0 -5826,0 -5827,0 -5828,0 -5829,0 -5830,0 -5831,0 -5832,0 -5833,0 -5834,0 -5835,0 -5836,0 -5837,0 -5838,0 -5839,0 -5840,0 -5841,0 -5842,0 -5843,0 -5844,0 -5845,0 -5846,0 -5847,0 -5848,0 -5849,0 -5850,0 -5851,0 -5852,0 -5853,0 -5854,0 -5855,0 -5856,0 -5857,0 -5858,0 -5859,0 -5860,0 -5861,0 -5862,0 -5863,0 -5864,0 -5865,0 -5866,0 -5867,0 -5868,0 -5869,0 -5870,0 -5871,0 -5872,0 -5873,0 -5874,0 -5875,0 -5876,0 -5877,0 -5878,0 -5879,0 -5880,0 -5881,0 -5882,0 -5883,0 -5884,0 -5885,0 -5886,0 -5887,0 -5888,0 -5889,0 -5890,0 -5891,0 -5892,0 -5893,0 -5894,0 -5895,0 -5896,0 -5897,0 -5898,0 -5899,0 -5900,0 -5901,0 -5902,0 -5903,0 -5904,0 -5905,0 -5906,0 -5907,0 -5908,0 -5909,0 -5910,0 -5911,0 -5912,0 -5913,0 -5914,0 -5915,0 -5916,0 -5917,0 -5918,0 -5919,0 -5920,0 -5921,0 -5922,0 -5923,0 -5924,0 -5925,0 -5926,0 -5927,0 -5928,0 -5929,0 -5930,0 -5931,0 -5932,0 -5933,0 -5934,0 -5935,0 -5936,0 -5937,0 -5938,0 -5939,0 -5940,0 -5941,0 -5942,0 -5943,0 -5944,0 -5945,0 -5946,0 -5947,0 -5948,0 -5949,0 -5950,0 -5951,0 -5952,0 -5953,0 -5954,0 -5955,0 -5956,0 -5957,0 -5958,0 -5959,0 -5960,0 -5961,0 -5962,0 -5963,0 -5964,0 -5965,0 -5966,0 -5967,0 -5968,0 -5969,0 -5970,0 -5971,0 -5972,0 -5973,0 -5974,0 -5975,0 -5976,0 -5977,0 -5978,0 -5979,0 -5980,0 -5981,0 -5982,0 -5983,0 -5984,0 -5985,0 -5986,0 -5987,0 -5988,0 -5989,0 -5990,0 -5991,0 -5992,0 -5993,0 -5994,0 -5995,0 -5996,0 -5997,0 -5998,0 -5999,0 -6000,0 -6001,0 -6002,0 -6003,0 -6004,0 -6005,0 -6006,0 -6007,0 -6008,0 -6009,0 -6010,0 -6011,0 -6012,0 -6013,0 -6014,0 -6015,0 -6016,0 -6017,0 -6018,0 -6019,0 -6020,0 -6021,0 -6022,0 -6023,0 -6024,0 -6025,0 -6026,0 -6027,0 -6028,0 -6029,0 -6030,0 -6031,0 -6032,0 -6033,0 -6034,0 -6035,0 -6036,0 -6037,0 -6038,0 -6039,0 -6040,0 -6041,0 -6042,0 -6043,0 -6044,0 -6045,0 -6046,0 -6047,0 -6048,0 -6049,0 -6050,0 -6051,0 -6052,0 -6053,0 -6054,0 -6055,0 -6056,0 -6057,0 -6058,0 -6059,0 -6060,0 -6061,0 -6062,0 -6063,0 -6064,0 -6065,0 -6066,0 -6067,0 -6068,0 -6069,0 -6070,0 -6071,0 -6072,0 -6073,0 -6074,0 -6075,0 -6076,0 -6077,0 -6078,0 -6079,0 -6080,0 -6081,0 -6082,0 -6083,0 -6084,0 -6085,0 -6086,0 -6087,0 -6088,0 -6089,0 -6090,0 -6091,0 -6092,0 -6093,0 -6094,0 -6095,0 -6096,0 -6097,0 -6098,0 -6099,0 -6100,0 -6101,0 -6102,0 -6103,0 -6104,0 -6105,0 -6106,0 -6107,0 -6108,0 -6109,0 -6110,0 -6111,0 -6112,0 -6113,0 -6114,0 -6115,0 -6116,0 -6117,0 -6118,0 -6119,0 -6120,0 -6121,0 -6122,0 -6123,0 -6124,0 -6125,0 -6126,0 -6127,0 -6128,0 -6129,0 -6130,0 -6131,0 -6132,0 -6133,0 -6134,0 -6135,0 -6136,0 -6137,0 -6138,0 -6139,0 -6140,0 -6141,0 -6142,0 -6143,0 -6144,0 -6145,0 -6146,0 -6147,0 -6148,0 -6149,0 -6150,0 -6151,0 -6152,0 -6153,0 -6154,0 -6155,0 -6156,0 -6157,0 -6158,0 -6159,0 -6160,0 -6161,0 -6162,0 -6163,0 -6164,0 -6165,0 -6166,0 -6167,0 -6168,0 -6169,0 -6170,0 -6171,0 -6172,0 -6173,0 -6174,0 -6175,0 -6176,0 -6177,0 -6178,0 -6179,0 -6180,0 -6181,0 -6182,0 -6183,0 -6184,0 -6185,0 -6186,0 -6187,0 -6188,0 -6189,0 -6190,0 -6191,0 -6192,0 -6193,0 -6194,0 -6195,0 -6196,0 -6197,0 -6198,0 -6199,0 -6200,0 -6201,0 -6202,0 -6203,0 -6204,0 -6205,0 -6206,0 -6207,0 -6208,0 -6209,0 -6210,0 -6211,0 -6212,0 -6213,0 -6214,0 -6215,0 -6216,0 -6217,0 -6218,0 -6219,0 -6220,0 -6221,0 -6222,0 -6223,0 -6224,0 -6225,0 -6226,0 -6227,0 -6228,0 -6229,0 -6230,0 -6231,0 -6232,0 -6233,0 -6234,0 -6235,0 -6236,0 -6237,0 -6238,0 -6239,0 -6240,0 -6241,0 -6242,0 -6243,0 -6244,0 -6245,0 -6246,0 -6247,0 -6248,0 -6249,0 -6250,0 -6251,0 -6252,0 -6253,0 -6254,0 -6255,0 -6256,0 -6257,0 -6258,0 -6259,0 -6260,0 -6261,0 -6262,0 -6263,0 -6264,0 -6265,0 -6266,0 -6267,0 -6268,0 -6269,0 -6270,0 -6271,0 -6272,0 -6273,0 -6274,0 -6275,0 -6276,0 -6277,0 -6278,0 -6279,0 -6280,0 -6281,0 -6282,0 -6283,0 -6284,0 -6285,0 -6286,0 -6287,0 -6288,0 -6289,0 -6290,0 -6291,0 -6292,0 -6293,0 -6294,0 -6295,0 -6296,0 -6297,0 -6298,0 -6299,0 -6300,0 -6301,0 -6302,0 -6303,0 -6304,0 -6305,0 -6306,0 -6307,0 -6308,0 -6309,0 -6310,0 -6311,0 -6312,0 -6313,0 -6314,0 -6315,0 -6316,0 -6317,0 -6318,0 -6319,0 -6320,0 -6321,0 -6322,0 -6323,0 -6324,0 -6325,0 -6326,0 -6327,0 -6328,0 -6329,0 -6330,0 -6331,0 -6332,0 -6333,0 -6334,0 -6335,0 -6336,0 -6337,0 -6338,0 -6339,0 -6340,0 -6341,0 -6342,0 -6343,0 -6344,0 -6345,0 -6346,0 -6347,0 -6348,0 -6349,0 -6350,0 -6351,0 -6352,0 -6353,0 -6354,0 -6355,0 -6356,0 -6357,0 -6358,0 -6359,0 -6360,0 -6361,0 -6362,0 -6363,0 -6364,0 -6365,0 -6366,0 -6367,0 -6368,0 -6369,0 -6370,0 -6371,0 -6372,0 -6373,0 -6374,0 -6375,0 -6376,0 -6377,0 -6378,0 -6379,0 -6380,0 -6381,0 -6382,0 -6383,0 -6384,0 -6385,0 -6386,0 -6387,0 -6388,0 -6389,0 -6390,0 -6391,0 -6392,0 -6393,0 -6394,0 -6395,0 -6396,0 -6397,0 -6398,0 -6399,0 -6400,0 -6401,0 -6402,0 -6403,0 -6404,0 -6405,0 -6406,0 -6407,0 -6408,0 -6409,0 -6410,0 -6411,0 -6412,0 -6413,0 -6414,0 -6415,0 -6416,0 -6417,0 -6418,0 -6419,0 -6420,0 -6421,0 -6422,0 -6423,0 -6424,0 -6425,0 -6426,0 -6427,0 -6428,0 -6429,0 -6430,0 -6431,0 -6432,0 -6433,0 -6434,0 -6435,0 -6436,0 -6437,0 -6438,0 -6439,0 -6440,0 -6441,0 -6442,0 -6443,0 -6444,0 -6445,0 -6446,0 -6447,0 -6448,0 -6449,0 -6450,0 -6451,0 -6452,0 -6453,0 -6454,0 -6455,0 -6456,0 -6457,0 -6458,0 -6459,0 -6460,0 -6461,0 -6462,0 -6463,0 -6464,0 -6465,0 -6466,0 -6467,0 -6468,0 -6469,0 -6470,0 -6471,0 -6472,0 -6473,0 -6474,0 -6475,0 -6476,0 -6477,0 -6478,0 -6479,0 -6480,0 -6481,0 -6482,0 -6483,0 -6484,0 -6485,0 -6486,0 -6487,0 -6488,0 -6489,0 -6490,0 -6491,0 -6492,0 -6493,0 -6494,0 -6495,0 -6496,0 -6497,0 -6498,0 -6499,0 -6500,0 -6501,0 -6502,0 -6503,0 -6504,0 -6505,0 -6506,0 -6507,0 -6508,0 -6509,0 -6510,0 -6511,0 -6512,0 -6513,0 -6514,0 -6515,0 -6516,0 -6517,0 -6518,0 -6519,0 -6520,0 -6521,0 -6522,0 -6523,0 -6524,0 -6525,0 -6526,0 -6527,0 -6528,0 -6529,0 -6530,0 -6531,0 -6532,0 -6533,0 -6534,0 -6535,0 -6536,0 -6537,0 -6538,0 -6539,0 -6540,0 -6541,0 -6542,0 -6543,0 -6544,0 -6545,0 -6546,0 -6547,0 -6548,0 -6549,0 -6550,0 -6551,0 -6552,0 -6553,0 -6554,0 -6555,0 -6556,0 -6557,0 -6558,0 -6559,0 -6560,0 -6561,0 -6562,0 -6563,0 -6564,0 -6565,0 -6566,0 -6567,0 -6568,0 -6569,0 -6570,0 -6571,0 -6572,0 -6573,0 -6574,0 -6575,0 -6576,0 -6577,0 -6578,0 -6579,0 -6580,0 -6581,0 -6582,0 -6583,0 -6584,0 -6585,0 -6586,0 -6587,0 -6588,0 -6589,0 -6590,0 -6591,0 -6592,0 -6593,0 -6594,0 -6595,0 -6596,0 -6597,0 -6598,0 -6599,0 -6600,0 -6601,0 -6602,0 -6603,0 -6604,0 -6605,0 -6606,0 -6607,0 -6608,0 -6609,0 -6610,0 -6611,0 -6612,0 -6613,0 -6614,0 -6615,0 -6616,0 -6617,0 -6618,0 -6619,0 -6620,0 -6621,0 -6622,0 -6623,0 -6624,0 -6625,0 -6626,0 -6627,0 -6628,0 -6629,0 -6630,0 -6631,0 -6632,0 -6633,0 -6634,0 -6635,0 -6636,0 -6637,0 -6638,0 -6639,0 -6640,0 -6641,0 -6642,0 -6643,0 -6644,0 -6645,0 -6646,0 -6647,0 -6648,0 -6649,0 -6650,0 -6651,0 -6652,0 -6653,0 -6654,0 -6655,0 -6656,0 -6657,0 -6658,0 -6659,0 -6660,0 -6661,0 -6662,0 -6663,0 -6664,0 -6665,0 -6666,0 -6667,0 -6668,0 -6669,0 -6670,0 -6671,0 -6672,0 -6673,0 -6674,0 -6675,0 -6676,0 -6677,0 -6678,0 -6679,0 -6680,0 -6681,0 -6682,0 -6683,0 -6684,0 -6685,0 -6686,0 -6687,0 -6688,0 -6689,0 -6690,0 -6691,0 -6692,0 -6693,0 -6694,0 -6695,0 -6696,0 -6697,0 -6698,0 -6699,0 -6700,0 -6701,0 -6702,0 -6703,0 -6704,0 -6705,0 -6706,0 -6707,0 -6708,0 -6709,0 -6710,0 -6711,0 -6712,0 -6713,0 -6714,0 -6715,0 -6716,0 -6717,0 -6718,0 -6719,0 -6720,0 -6721,0 -6722,0 -6723,0 -6724,0 -6725,0 -6726,0 -6727,0 -6728,0 -6729,0 -6730,0 -6731,0 -6732,0 -6733,0 -6734,0 -6735,0 -6736,0 -6737,0 -6738,0 -6739,0 -6740,0 -6741,0 -6742,0 -6743,0 -6744,0 -6745,0 -6746,0 -6747,0 -6748,0 -6749,0 -6750,0 -6751,0 -6752,0 -6753,0 -6754,0 -6755,0 -6756,0 -6757,0 -6758,0 -6759,0 -6760,0 -6761,0 -6762,0 -6763,0 -6764,0 -6765,0 -6766,0 -6767,0 -6768,0 -6769,0 -6770,0 -6771,0 -6772,0 -6773,0 -6774,0 -6775,0 -6776,0 -6777,0 -6778,0 -6779,0 -6780,0 -6781,0 -6782,0 -6783,0 -6784,0 -6785,0 -6786,0 -6787,0 -6788,0 -6789,0 -6790,0 -6791,0 -6792,0 -6793,0 -6794,0 -6795,0 -6796,0 -6797,0 -6798,0 -6799,0 -6800,0 -6801,0 -6802,0 -6803,0 -6804,0 -6805,0 -6806,0 -6807,0 -6808,0 -6809,0 -6810,0 -6811,0 -6812,0 -6813,0 -6814,0 -6815,0 -6816,0 -6817,0 -6818,0 -6819,0 -6820,0 -6821,0 -6822,0 -6823,0 -6824,0 -6825,0 -6826,0 -6827,0 -6828,0 -6829,0 -6830,0 -6831,0 -6832,0 -6833,0 -6834,0 -6835,0 -6836,0 -6837,0 -6838,0 -6839,0 -6840,0 -6841,0 -6842,0 -6843,0 -6844,0 -6845,0 -6846,0 -6847,0 -6848,0 -6849,0 -6850,0 -6851,0 -6852,0 -6853,0 -6854,0 -6855,0 -6856,0 -6857,0 -6858,0 -6859,0 -6860,0 -6861,0 -6862,0 -6863,0 -6864,0 -6865,0 -6866,0 -6867,0 -6868,0 -6869,0 -6870,0 -6871,0 -6872,0 -6873,0 -6874,0 -6875,0 -6876,0 -6877,0 -6878,0 -6879,0 -6880,0 -6881,0 -6882,0 -6883,0 -6884,0 -6885,0 -6886,0 -6887,0 -6888,0 -6889,0 -6890,0 -6891,0 -6892,0 -6893,0 -6894,0 -6895,0 -6896,0 -6897,0 -6898,0 -6899,0 -6900,0 -6901,0 -6902,0 -6903,0 -6904,0 -6905,0 -6906,0 -6907,0 -6908,0 -6909,0 -6910,0 -6911,0 -6912,0 -6913,0 -6914,0 -6915,0 -6916,0 -6917,0 -6918,0 -6919,0 -6920,0 -6921,0 -6922,0 -6923,0 -6924,0 -6925,0 -6926,0 -6927,0 -6928,0 -6929,0 -6930,0 -6931,0 -6932,0 -6933,0 -6934,0 -6935,0 -6936,0 -6937,0 -6938,0 -6939,0 -6940,0 -6941,0 -6942,0 -6943,0 -6944,0 -6945,0 -6946,0 -6947,0 -6948,0 -6949,0 -6950,0 -6951,0 -6952,0 -6953,0 -6954,0 -6955,0 -6956,0 -6957,0 -6958,0 -6959,0 -6960,0 -6961,0 -6962,0 -6963,0 -6964,0 -6965,0 -6966,0 -6967,0 -6968,0 -6969,0 -6970,0 -6971,0 -6972,0 -6973,0 -6974,0 -6975,0 -6976,0 -6977,0 -6978,0 -6979,0 -6980,0 -6981,0 -6982,0 -6983,0 -6984,0 -6985,0 -6986,0 -6987,0 -6988,0 -6989,0 -6990,0 -6991,0 -6992,0 -6993,0 -6994,0 -6995,0 -6996,0 -6997,0 -6998,0 -6999,0 -7000,0 -7001,0 -7002,0 -7003,0 -7004,0 -7005,0 -7006,0 -7007,0 -7008,0 -7009,0 -7010,0 -7011,0 -7012,0 -7013,0 -7014,0 -7015,0 -7016,0 -7017,0 -7018,0 -7019,0 -7020,0 -7021,0 -7022,0 -7023,0 -7024,0 -7025,0 -7026,0 -7027,0 -7028,0 -7029,0 -7030,0 -7031,0 -7032,0 -7033,0 -7034,0 -7035,0 -7036,0 -7037,0 -7038,0 -7039,0 -7040,0 -7041,0 -7042,0 -7043,0 -7044,0 -7045,0 -7046,0 -7047,0 -7048,0 -7049,0 -7050,0 -7051,0 -7052,0 -7053,0 -7054,0 -7055,0 -7056,0 -7057,0 -7058,0 -7059,0 -7060,0 -7061,0 -7062,0 -7063,0 -7064,0 -7065,0 -7066,0 -7067,0 -7068,0 -7069,0 -7070,0 -7071,0 -7072,0 -7073,0 -7074,0 -7075,0 -7076,0 -7077,0 -7078,0 -7079,0 -7080,0 -7081,0 -7082,0 -7083,0 -7084,0 -7085,0 -7086,0 -7087,0 -7088,0 -7089,0 -7090,0 -7091,0 -7092,0 -7093,0 -7094,0 -7095,0 -7096,0 -7097,0 -7098,0 -7099,0 -7100,0 -7101,0 -7102,0 -7103,0 -7104,0 -7105,0 -7106,0 -7107,0 -7108,0 -7109,0 -7110,0 -7111,0 -7112,0 -7113,0 -7114,0 -7115,0 -7116,0 -7117,0 -7118,0 -7119,0 -7120,0 -7121,0 -7122,0 -7123,0 -7124,0 -7125,0 -7126,0 -7127,0 -7128,0 -7129,0 -7130,0 -7131,0 -7132,0 -7133,0 -7134,0 -7135,0 -7136,0 -7137,0 -7138,0 -7139,0 -7140,0 -7141,0 -7142,0 -7143,0 -7144,0 -7145,0 -7146,0 -7147,0 -7148,0 -7149,0 -7150,0 -7151,0 -7152,0 -7153,0 -7154,0 -7155,0 -7156,0 -7157,0 -7158,0 -7159,0 -7160,0 -7161,0 -7162,0 -7163,0 -7164,0 -7165,0 -7166,0 -7167,0 -7168,0 -7169,0 -7170,0 -7171,0 -7172,0 -7173,0 -7174,0 -7175,0 -7176,0 -7177,0 -7178,0 -7179,0 -7180,0 -7181,0 -7182,0 -7183,0 -7184,0 -7185,0 -7186,0 -7187,0 -7188,0 -7189,0 -7190,0 -7191,0 -7192,0 -7193,0 -7194,0 -7195,0 -7196,0 -7197,0 -7198,0 -7199,0 -7200,0 -7201,0 -7202,0 -7203,0 -7204,0 -7205,0 -7206,0 -7207,0 -7208,0 -7209,0 -7210,0 -7211,0 -7212,0 -7213,0 -7214,0 -7215,0 -7216,0 -7217,0 -7218,0 -7219,0 -7220,0 -7221,0 -7222,0 -7223,0 -7224,0 -7225,0 -7226,0 -7227,0 -7228,0 -7229,0 -7230,0 -7231,0 -7232,0 -7233,0 -7234,0 -7235,0 -7236,0 -7237,0 -7238,0 -7239,0 -7240,0 -7241,0 -7242,0 -7243,0 -7244,0 -7245,0 -7246,0 -7247,0 -7248,0 -7249,0 -7250,0 -7251,0 -7252,0 -7253,0 -7254,0 -7255,0 -7256,0 -7257,0 -7258,0 -7259,0 -7260,0 -7261,0 -7262,0 -7263,0 -7264,0 -7265,0 -7266,0 -7267,0 -7268,0 -7269,0 -7270,0 -7271,0 -7272,0 -7273,0 -7274,0 -7275,0 -7276,0 -7277,0 -7278,0 -7279,0 -7280,0 -7281,0 -7282,0 -7283,0 -7284,0 -7285,0 -7286,0 -7287,0 -7288,0 -7289,0 -7290,0 -7291,0 -7292,0 -7293,0 -7294,0 -7295,0 -7296,0 -7297,0 -7298,0 -7299,0 -7300,0 -7301,0 -7302,0 -7303,0 -7304,0 -7305,0 -7306,0 -7307,0 -7308,0 -7309,0 -7310,0 -7311,0 -7312,0 -7313,0 -7314,0 -7315,0 -7316,0 -7317,0 -7318,0 -7319,0 -7320,0 -7321,0 -7322,0 -7323,0 -7324,0 -7325,0 -7326,0 -7327,0 -7328,0 -7329,0 -7330,0 -7331,0 -7332,0 -7333,0 -7334,0 -7335,0 -7336,0 -7337,0 -7338,0 -7339,0 -7340,0 -7341,0 -7342,0 -7343,0 -7344,0 -7345,0 -7346,0 -7347,0 -7348,0 -7349,0 -7350,0 -7351,0 -7352,0 -7353,0 -7354,0 -7355,0 -7356,0 -7357,0 -7358,0 -7359,0 -7360,0 -7361,0 -7362,0 -7363,0 -7364,0 -7365,0 -7366,0 -7367,0 -7368,0 -7369,0 -7370,0 -7371,0 -7372,0 -7373,0 -7374,0 -7375,0 -7376,0 -7377,0 -7378,0 -7379,0 -7380,0 -7381,0 -7382,0 -7383,0 -7384,0 -7385,0 -7386,0 -7387,0 -7388,0 -7389,0 -7390,0 -7391,0 -7392,0 -7393,0 -7394,0 -7395,0 -7396,0 -7397,0 -7398,0 -7399,0 -7400,0 -7401,0 -7402,0 -7403,0 -7404,0 -7405,0 -7406,0 -7407,0 -7408,0 -7409,0 -7410,0 -7411,0 -7412,0 -7413,0 -7414,0 -7415,0 -7416,0 -7417,0 -7418,0 -7419,0 -7420,0 -7421,0 -7422,0 -7423,0 -7424,0 -7425,0 -7426,0 -7427,0 -7428,0 -7429,0 -7430,0 -7431,0 -7432,0 -7433,0 -7434,0 -7435,0 -7436,0 -7437,0 -7438,0 -7439,0 -7440,0 -7441,0 -7442,0 -7443,0 -7444,0 -7445,0 -7446,0 -7447,0 -7448,0 -7449,0 -7450,0 -7451,0 -7452,0 -7453,0 -7454,0 -7455,0 -7456,0 -7457,0 -7458,0 -7459,0 -7460,0 -7461,0 -7462,0 -7463,0 -7464,0 -7465,0 -7466,0 -7467,0 -7468,0 -7469,0 -7470,0 -7471,0 -7472,0 -7473,0 -7474,0 -7475,0 -7476,0 -7477,0 -7478,0 -7479,0 -7480,0 -7481,0 -7482,0 -7483,0 -7484,0 -7485,0 -7486,0 -7487,0 -7488,0 -7489,0 -7490,0 -7491,0 -7492,0 -7493,0 -7494,0 -7495,0 -7496,0 -7497,0 -7498,0 -7499,0 -7500,0 -7501,0 -7502,0 -7503,0 -7504,0 -7505,0 -7506,0 -7507,0 -7508,0 -7509,0 -7510,0 -7511,0 -7512,0 -7513,0 -7514,0 -7515,0 -7516,0 -7517,0 -7518,0 -7519,0 -7520,0 -7521,0 -7522,0 -7523,0 -7524,0 -7525,0 -7526,0 -7527,0 -7528,0 -7529,0 -7530,0 -7531,0 -7532,0 -7533,0 -7534,0 -7535,0 -7536,0 -7537,0 -7538,0 -7539,0 -7540,0 -7541,0 -7542,0 -7543,0 -7544,0 -7545,0 -7546,0 -7547,0 -7548,0 -7549,0 -7550,0 -7551,0 -7552,0 -7553,0 -7554,0 -7555,0 -7556,0 -7557,0 -7558,0 -7559,0 -7560,0 -7561,0 -7562,0 -7563,0 -7564,0 -7565,0 -7566,0 -7567,0 -7568,0 -7569,0 -7570,0 -7571,0 -7572,0 -7573,0 -7574,0 -7575,0 -7576,0 -7577,0 -7578,0 -7579,0 -7580,0 -7581,0 -7582,0 -7583,0 -7584,0 -7585,0 -7586,0 -7587,0 -7588,0 -7589,0 -7590,0 -7591,0 -7592,0 -7593,0 -7594,0 -7595,0 -7596,0 -7597,0 -7598,0 -7599,0 -7600,0 -7601,0 -7602,0 -7603,0 -7604,0 -7605,0 -7606,0 -7607,0 -7608,0 -7609,0 -7610,0 -7611,0 -7612,0 -7613,0 -7614,0 -7615,0 -7616,0 -7617,0 -7618,0 -7619,0 -7620,0 -7621,0 -7622,0 -7623,0 -7624,0 -7625,0 -7626,0 -7627,0 -7628,0 -7629,0 -7630,0 -7631,0 -7632,0 -7633,0 -7634,0 -7635,0 -7636,0 -7637,0 -7638,0 -7639,0 -7640,0 -7641,0 -7642,0 -7643,0 -7644,0 -7645,0 -7646,0 -7647,0 -7648,0 -7649,0 -7650,0 -7651,0 -7652,0 -7653,0 -7654,0 -7655,0 -7656,0 -7657,0 -7658,0 -7659,0 -7660,0 -7661,0 -7662,0 -7663,0 -7664,0 -7665,0 -7666,0 -7667,0 -7668,0 -7669,0 -7670,0 -7671,0 -7672,0 -7673,0 -7674,0 -7675,0 -7676,0 -7677,0 -7678,0 -7679,0 -7680,0 -7681,0 -7682,0 -7683,0 -7684,0 -7685,0 -7686,0 -7687,0 -7688,0 -7689,0 -7690,0 -7691,0 -7692,0 -7693,0 -7694,0 -7695,0 -7696,0 -7697,0 -7698,0 -7699,0 -7700,0 -7701,0 -7702,0 -7703,0 -7704,0 -7705,0 -7706,0 -7707,0 -7708,0 -7709,0 -7710,0 -7711,0 -7712,0 -7713,0 -7714,0 -7715,0 -7716,0 -7717,0 -7718,0 -7719,0 -7720,0 -7721,0 -7722,0 -7723,0 -7724,0 -7725,0 -7726,0 -7727,0 -7728,0 -7729,0 -7730,0 -7731,0 -7732,0 -7733,0 -7734,0 -7735,0 -7736,0 -7737,0 -7738,0 -7739,0 -7740,0 -7741,0 -7742,0 -7743,0 -7744,0 -7745,0 -7746,0 -7747,0 -7748,0 -7749,0 -7750,0 -7751,0 -7752,0 -7753,0 -7754,0 -7755,0 -7756,0 -7757,0 -7758,0 -7759,0 -7760,0 -7761,0 -7762,0 -7763,0 -7764,0 -7765,0 -7766,0 -7767,0 -7768,0 -7769,0 -7770,0 -7771,0 -7772,0 -7773,0 -7774,0 -7775,0 -7776,0 -7777,0 -7778,0 -7779,0 -7780,0 -7781,0 -7782,0 -7783,0 -7784,0 -7785,0 -7786,0 -7787,0 -7788,0 -7789,0 -7790,0 -7791,0 -7792,0 -7793,0 -7794,0 -7795,0 -7796,0 -7797,0 -7798,0 -7799,0 -7800,0 -7801,0 -7802,0 -7803,0 -7804,0 -7805,0 -7806,0 -7807,0 -7808,0 -7809,0 -7810,0 -7811,0 -7812,0 -7813,0 -7814,0 -7815,0 -7816,0 -7817,0 -7818,0 -7819,0 -7820,0 -7821,0 -7822,0 -7823,0 -7824,0 -7825,0 -7826,0 -7827,0 -7828,0 -7829,0 -7830,0 -7831,0 -7832,0 -7833,0 -7834,0 -7835,0 -7836,0 -7837,0 -7838,0 -7839,0 -7840,0 -7841,0 -7842,0 -7843,0 -7844,0 -7845,0 -7846,0 -7847,0 -7848,0 -7849,0 -7850,0 -7851,0 -7852,0 -7853,0 -7854,0 -7855,0 -7856,0 -7857,0 -7858,0 -7859,0 -7860,0 -7861,0 -7862,0 -7863,0 -7864,0 -7865,0 -7866,0 -7867,0 -7868,0 -7869,0 -7870,0 -7871,0 -7872,0 -7873,0 -7874,0 -7875,0 -7876,0 -7877,0 -7878,0 -7879,0 -7880,0 -7881,0 -7882,0 -7883,0 -7884,0 -7885,0 -7886,0 -7887,0 -7888,0 -7889,0 -7890,0 -7891,0 -7892,0 -7893,0 -7894,0 -7895,0 -7896,0 -7897,0 -7898,0 -7899,0 -7900,0 -7901,0 -7902,0 -7903,0 -7904,0 -7905,0 -7906,0 -7907,0 -7908,0 -7909,0 -7910,0 -7911,0 -7912,0 -7913,0 -7914,0 -7915,0 -7916,0 -7917,0 -7918,0 -7919,0 -7920,0 -7921,0 -7922,0 -7923,0 -7924,0 -7925,0 -7926,0 -7927,0 -7928,0 -7929,0 -7930,0 -7931,0 -7932,0 -7933,0 -7934,0 -7935,0 -7936,0 -7937,0 -7938,0 -7939,0 -7940,0 -7941,0 -7942,0 -7943,0 -7944,0 -7945,0 -7946,0 -7947,0 -7948,0 -7949,0 -7950,0 -7951,0 -7952,0 -7953,0 -7954,0 -7955,0 -7956,0 -7957,0 -7958,0 -7959,0 -7960,0 -7961,0 -7962,0 -7963,0 -7964,0 -7965,0 -7966,0 -7967,0 -7968,0 -7969,0 -7970,0 -7971,0 -7972,0 -7973,0 -7974,0 -7975,0 -7976,0 -7977,0 -7978,0 -7979,0 -7980,0 -7981,0 -7982,0 -7983,0 -7984,0 -7985,0 -7986,0 -7987,0 -7988,0 -7989,0 -7990,0 -7991,0 -7992,0 -7993,0 -7994,0 -7995,0 -7996,0 -7997,0 -7998,0 -7999,0 -8000,0 -8001,0 -8002,0 -8003,0 -8004,0 -8005,0 -8006,0 -8007,0 -8008,0 -8009,0 -8010,0 -8011,0 -8012,0 -8013,0 -8014,0 -8015,0 -8016,0 -8017,0 -8018,0 -8019,0 -8020,0 -8021,0 -8022,0 -8023,0 -8024,0 -8025,0 -8026,0 -8027,0 -8028,0 -8029,0 -8030,0 -8031,0 -8032,0 -8033,0 -8034,0 -8035,0 -8036,0 -8037,0 -8038,0 -8039,0 -8040,0 -8041,0 -8042,0 -8043,0 -8044,0 -8045,0 -8046,0 -8047,0 -8048,0 -8049,0 -8050,0 -8051,0 -8052,0 -8053,0 -8054,0 -8055,0 -8056,0 -8057,0 -8058,0 -8059,0 -8060,0 -8061,0 -8062,0 -8063,0 -8064,0 -8065,0 -8066,0 -8067,0 -8068,0 -8069,0 -8070,0 -8071,0 -8072,0 -8073,0 -8074,0 -8075,0 -8076,0 -8077,0 -8078,0 -8079,0 -8080,0 -8081,0 -8082,0 -8083,0 -8084,0 -8085,0 -8086,0 -8087,0 -8088,0 -8089,0 -8090,0 -8091,0 -8092,0 -8093,0 -8094,0 -8095,0 -8096,0 -8097,0 -8098,0 -8099,0 -8100,0 -8101,0 -8102,0 -8103,0 -8104,0 -8105,0 -8106,0 -8107,0 -8108,0 -8109,0 -8110,0 -8111,0 -8112,0 -8113,0 -8114,0 -8115,0 -8116,0 -8117,0 -8118,0 -8119,0 -8120,0 -8121,0 -8122,0 -8123,0 -8124,0 -8125,0 -8126,0 -8127,0 -8128,0 -8129,0 -8130,0 -8131,0 -8132,0 -8133,0 -8134,0 -8135,0 -8136,0 -8137,0 -8138,0 -8139,0 -8140,0 -8141,0 -8142,0 -8143,0 -8144,0 -8145,0 -8146,0 -8147,0 -8148,0 -8149,0 -8150,0 -8151,0 -8152,0 -8153,0 -8154,0 -8155,0 -8156,0 -8157,0 -8158,0 -8159,0 -8160,0 -8161,0 -8162,0 -8163,0 -8164,0 -8165,0 -8166,0 -8167,0 -8168,0 -8169,0 -8170,0 -8171,0 -8172,0 -8173,0 -8174,0 -8175,0 -8176,0 -8177,0 -8178,0 -8179,0 -8180,0 -8181,0 -8182,0 -8183,0 -8184,0 -8185,0 -8186,0 -8187,0 -8188,0 -8189,0 -8190,0 -8191,0 -8192,0 -8193,0 -8194,0 -8195,0 -8196,0 -8197,0 -8198,0 -8199,0 -8200,0 -8201,0 -8202,0 -8203,0 -8204,0 -8205,0 -8206,0 -8207,0 -8208,0 -8209,0 -8210,0 -8211,0 -8212,0 -8213,0 -8214,0 -8215,0 -8216,0 -8217,0 -8218,0 -8219,0 -8220,0 -8221,0 -8222,0 -8223,0 -8224,0 -8225,0 -8226,0 -8227,0 -8228,0 -8229,0 -8230,0 -8231,0 -8232,0 -8233,0 -8234,0 -8235,0 -8236,0 -8237,0 -8238,0 -8239,0 -8240,0 -8241,0 -8242,0 -8243,0 -8244,0 -8245,0 -8246,0 -8247,0 -8248,0 -8249,0 -8250,0 -8251,0 -8252,0 -8253,0 -8254,0 -8255,0 -8256,0 -8257,0 -8258,0 -8259,0 -8260,0 -8261,0 -8262,0 -8263,0 -8264,0 -8265,0 -8266,0 -8267,0 -8268,0 -8269,0 -8270,0 -8271,0 -8272,0 -8273,0 -8274,0 -8275,0 -8276,0 -8277,0 -8278,0 -8279,0 -8280,0 -8281,0 -8282,0 -8283,0 -8284,0 -8285,0 -8286,0 -8287,0 -8288,0 -8289,0 -8290,0 -8291,0 -8292,0 -8293,0 -8294,0 -8295,0 -8296,0 -8297,0 -8298,0 -8299,0 -8300,0 -8301,0 -8302,0 -8303,0 -8304,0 -8305,0 -8306,0 -8307,0 -8308,0 -8309,0 -8310,0 -8311,0 -8312,0 -8313,0 -8314,0 -8315,0 -8316,0 -8317,0 -8318,0 -8319,0 -8320,0 -8321,0 -8322,0 -8323,0 -8324,0 -8325,0 -8326,0 -8327,0 -8328,0 -8329,0 -8330,0 -8331,0 -8332,0 -8333,0 -8334,0 -8335,0 -8336,0 -8337,0 -8338,0 -8339,0 -8340,0 -8341,0 -8342,0 -8343,0 -8344,0 -8345,0 -8346,0 -8347,0 -8348,0 -8349,0 -8350,0 -8351,0 -8352,0 -8353,0 -8354,0 -8355,0 -8356,0 -8357,0 -8358,0 -8359,0 -8360,0 -8361,0 -8362,0 -8363,0 -8364,0 -8365,0 -8366,0 -8367,0 -8368,0 -8369,0 -8370,0 -8371,0 -8372,0 -8373,0 -8374,0 -8375,0 -8376,0 -8377,0 -8378,0 -8379,0 -8380,0 -8381,0 -8382,0 -8383,0 -8384,0 -8385,0 -8386,0 -8387,0 -8388,0 -8389,0 -8390,0 -8391,0 -8392,0 -8393,0 -8394,0 -8395,0 -8396,0 -8397,0 -8398,0 -8399,0 -8400,0 -8401,0 -8402,0 -8403,0 -8404,0 -8405,0 -8406,0 -8407,0 -8408,0 -8409,0 -8410,0 -8411,0 -8412,0 -8413,0 -8414,0 -8415,0 -8416,0 -8417,0 -8418,0 -8419,0 -8420,0 -8421,0 -8422,0 -8423,0 -8424,0 -8425,0 -8426,0 -8427,0 -8428,0 -8429,0 -8430,0 -8431,0 -8432,0 -8433,0 -8434,0 -8435,0 -8436,0 -8437,0 -8438,0 -8439,0 -8440,0 -8441,0 -8442,0 -8443,0 -8444,0 -8445,0 -8446,0 -8447,0 -8448,0 -8449,0 -8450,0 -8451,0 -8452,0 -8453,0 -8454,0 -8455,0 -8456,0 -8457,0 -8458,0 -8459,0 -8460,0 -8461,0 -8462,0 -8463,0 -8464,0 -8465,0 -8466,0 -8467,0 -8468,0 -8469,0 -8470,0 -8471,0 -8472,0 -8473,0 -8474,0 -8475,0 -8476,0 -8477,0 -8478,0 -8479,0 -8480,0 -8481,0 -8482,0 -8483,0 -8484,0 -8485,0 -8486,0 -8487,0 -8488,0 -8489,0 -8490,0 -8491,0 -8492,0 -8493,0 -8494,0 -8495,0 -8496,0 -8497,0 -8498,0 -8499,0 -8500,0 -8501,0 -8502,0 -8503,0 -8504,0 -8505,0 -8506,0 -8507,0 -8508,0 -8509,0 -8510,0 -8511,0 -8512,0 -8513,0 -8514,0 -8515,0 -8516,0 -8517,0 -8518,0 -8519,0 -8520,0 -8521,0 -8522,0 -8523,0 -8524,0 -8525,0 -8526,0 -8527,0 -8528,0 -8529,0 -8530,0 -8531,0 -8532,0 -8533,0 -8534,0 -8535,0 -8536,0 -8537,0 -8538,0 -8539,0 -8540,0 -8541,0 -8542,0 -8543,0 -8544,0 -8545,0 -8546,0 -8547,0 -8548,0 -8549,0 -8550,0 -8551,0 -8552,0 -8553,0 -8554,0 -8555,0 -8556,0 -8557,0 -8558,0 -8559,0 -8560,0 -8561,0 -8562,0 -8563,0 -8564,0 -8565,0 -8566,0 -8567,0 -8568,0 -8569,0 -8570,0 -8571,0 -8572,0 -8573,0 -8574,0 -8575,0 -8576,0 -8577,0 -8578,0 -8579,0 -8580,0 -8581,0 -8582,0 -8583,0 -8584,0 -8585,0 -8586,0 -8587,0 -8588,0 -8589,0 -8590,0 -8591,0 -8592,0 -8593,0 -8594,0 -8595,0 -8596,0 -8597,0 -8598,0 -8599,0 -8600,0 -8601,0 -8602,0 -8603,0 -8604,0 -8605,0 -8606,0 -8607,0 -8608,0 -8609,0 -8610,0 -8611,0 -8612,0 -8613,0 -8614,0 -8615,0 -8616,0 -8617,0 -8618,0 -8619,0 -8620,0 -8621,0 -8622,0 -8623,0 -8624,0 -8625,0 -8626,0 -8627,0 -8628,0 -8629,0 -8630,0 -8631,0 -8632,0 -8633,0 -8634,0 -8635,0 -8636,0 -8637,0 -8638,0 -8639,0 -8640,0 -8641,0 -8642,0 -8643,0 -8644,0 -8645,0 -8646,0 -8647,0 -8648,0 -8649,0 -8650,0 -8651,0 -8652,0 -8653,0 -8654,0 -8655,0 -8656,0 -8657,0 -8658,0 -8659,0 -8660,0 -8661,0 -8662,0 -8663,0 -8664,0 -8665,0 -8666,0 -8667,0 -8668,0 -8669,0 -8670,0 -8671,0 -8672,0 -8673,0 -8674,0 -8675,0 -8676,0 -8677,0 -8678,0 -8679,0 -8680,0 -8681,0 -8682,0 -8683,0 -8684,0 -8685,0 -8686,0 -8687,0 -8688,0 -8689,0 -8690,0 -8691,0 -8692,0 -8693,0 -8694,0 -8695,0 -8696,0 -8697,0 -8698,0 -8699,0 -8700,0 -8701,0 -8702,0 -8703,0 -8704,0 -8705,0 -8706,0 -8707,0 -8708,0 -8709,0 -8710,0 -8711,0 -8712,0 -8713,0 -8714,0 -8715,0 -8716,0 -8717,0 -8718,0 -8719,0 -8720,0 -8721,0 -8722,0 -8723,0 -8724,0 -8725,0 -8726,0 -8727,0 -8728,0 -8729,0 -8730,0 -8731,0 -8732,0 -8733,0 -8734,0 -8735,0 -8736,0 -8737,0 -8738,0 -8739,0 -8740,0 -8741,0 -8742,0 -8743,0 -8744,0 -8745,0 -8746,0 -8747,0 -8748,0 -8749,0 -8750,0 -8751,0 -8752,0 -8753,0 -8754,0 -8755,0 -8756,0 -8757,0 -8758,0 -8759,0 -8760,0 -8761,0 -8762,0 -8763,0 -8764,0 -8765,0 -8766,0 -8767,0 -8768,0 -8769,0 -8770,0 -8771,0 -8772,0 -8773,0 -8774,0 -8775,0 -8776,0 -8777,0 -8778,0 -8779,0 -8780,0 -8781,0 -8782,0 -8783,0 -8784,0 -8785,0 -8786,0 -8787,0 -8788,0 -8789,0 -8790,0 -8791,0 -8792,0 -8793,0 -8794,0 -8795,0 -8796,0 -8797,0 -8798,0 -8799,0 -8800,0 -8801,0 -8802,0 -8803,0 -8804,0 -8805,0 -8806,0 -8807,0 -8808,0 -8809,0 -8810,0 -8811,0 -8812,0 -8813,0 -8814,0 -8815,0 -8816,0 -8817,0 -8818,0 -8819,0 -8820,0 -8821,0 -8822,0 -8823,0 -8824,0 -8825,0 -8826,0 -8827,0 -8828,0 -8829,0 -8830,0 -8831,0 -8832,0 -8833,0 -8834,0 -8835,0 -8836,0 -8837,0 -8838,0 -8839,0 -8840,0 -8841,0 -8842,0 -8843,0 -8844,0 -8845,0 -8846,0 -8847,0 -8848,0 -8849,0 -8850,0 -8851,0 -8852,0 -8853,0 -8854,0 -8855,0 -8856,0 -8857,0 -8858,0 -8859,0 -8860,0 -8861,0 -8862,0 -8863,0 -8864,0 -8865,0 -8866,0 -8867,0 -8868,0 -8869,0 -8870,0 -8871,0 -8872,0 -8873,0 -8874,0 -8875,0 -8876,0 -8877,0 -8878,0 -8879,0 -8880,0 -8881,0 -8882,0 -8883,0 -8884,0 -8885,0 -8886,0 -8887,0 -8888,0 -8889,0 -8890,0 -8891,0 -8892,0 -8893,0 -8894,0 -8895,0 -8896,0 -8897,0 -8898,0 -8899,0 -8900,0 -8901,0 -8902,0 -8903,0 -8904,0 -8905,0 -8906,0 -8907,0 -8908,0 -8909,0 -8910,0 -8911,0 -8912,0 -8913,0 -8914,0 -8915,0 -8916,0 -8917,0 -8918,0 -8919,0 -8920,0 -8921,0 -8922,0 -8923,0 -8924,0 -8925,0 -8926,0 -8927,0 -8928,0 -8929,0 -8930,0 -8931,0 -8932,0 -8933,0 -8934,0 -8935,0 -8936,0 -8937,0 -8938,0 -8939,0 -8940,0 -8941,0 -8942,0 -8943,0 -8944,0 -8945,0 -8946,0 -8947,0 -8948,0 -8949,0 -8950,0 -8951,0 -8952,0 -8953,0 -8954,0 -8955,0 -8956,0 -8957,0 -8958,0 -8959,0 -8960,0 -8961,0 -8962,0 -8963,0 -8964,0 -8965,0 -8966,0 -8967,0 -8968,0 -8969,0 -8970,0 -8971,0 -8972,0 -8973,0 -8974,0 -8975,0 -8976,0 -8977,0 -8978,0 -8979,0 -8980,0 -8981,0 -8982,0 -8983,0 -8984,0 -8985,0 -8986,0 -8987,0 -8988,0 -8989,0 -8990,0 -8991,0 -8992,0 -8993,0 -8994,0 -8995,0 -8996,0 -8997,0 -8998,0 -8999,0 -9000,0 -9001,0 -9002,0 -9003,0 -9004,0 -9005,0 -9006,0 -9007,0 -9008,0 -9009,0 -9010,0 -9011,0 -9012,0 -9013,0 -9014,0 -9015,0 -9016,0 -9017,0 -9018,0 -9019,0 -9020,0 -9021,0 -9022,0 -9023,0 -9024,0 -9025,0 -9026,0 -9027,0 -9028,0 -9029,0 -9030,0 -9031,0 -9032,0 -9033,0 -9034,0 -9035,0 -9036,0 -9037,0 -9038,0 -9039,0 -9040,0 -9041,0 -9042,0 -9043,0 -9044,0 -9045,0 -9046,0 -9047,0 -9048,0 -9049,0 -9050,0 -9051,0 -9052,0 -9053,0 -9054,0 -9055,0 -9056,0 -9057,0 -9058,0 -9059,0 -9060,0 -9061,0 -9062,0 -9063,0 -9064,0 -9065,0 -9066,0 -9067,0 -9068,0 -9069,0 -9070,0 -9071,0 -9072,0 -9073,0 -9074,0 -9075,0 -9076,0 -9077,0 -9078,0 -9079,0 -9080,0 -9081,0 -9082,0 -9083,0 -9084,0 -9085,0 -9086,0 -9087,0 -9088,0 -9089,0 -9090,0 -9091,0 -9092,0 -9093,0 -9094,0 -9095,0 -9096,0 -9097,0 -9098,0 -9099,0 -9100,0 -9101,0 -9102,0 -9103,0 -9104,0 -9105,0 -9106,0 -9107,0 -9108,0 -9109,0 -9110,0 -9111,0 -9112,0 -9113,0 -9114,0 -9115,0 -9116,0 -9117,0 -9118,0 -9119,0 -9120,0 -9121,0 -9122,0 -9123,0 -9124,0 -9125,0 -9126,0 -9127,0 -9128,0 -9129,0 -9130,0 -9131,0 -9132,0 -9133,0 -9134,0 -9135,0 -9136,0 -9137,0 -9138,0 -9139,0 -9140,0 -9141,0 -9142,0 -9143,0 -9144,0 -9145,0 -9146,0 -9147,0 -9148,0 -9149,0 -9150,0 -9151,0 -9152,0 -9153,0 -9154,0 -9155,0 -9156,0 -9157,0 -9158,0 -9159,0 -9160,0 -9161,0 -9162,0 -9163,0 -9164,0 -9165,0 -9166,0 -9167,0 -9168,0 -9169,0 -9170,0 -9171,0 -9172,0 -9173,0 -9174,0 -9175,0 -9176,0 -9177,0 -9178,0 -9179,0 -9180,0 -9181,0 -9182,0 -9183,0 -9184,0 -9185,0 -9186,0 -9187,0 -9188,0 -9189,0 -9190,0 -9191,0 -9192,0 -9193,0 -9194,0 -9195,0 -9196,0 -9197,0 -9198,0 -9199,0 -9200,0 -9201,0 -9202,0 -9203,0 -9204,0 -9205,0 -9206,0 -9207,0 -9208,0 -9209,0 -9210,0 -9211,0 -9212,0 -9213,0 -9214,0 -9215,0 -9216,0 -9217,0 -9218,0 -9219,0 -9220,0 -9221,0 -9222,0 -9223,0 -9224,0 -9225,0 -9226,0 -9227,0 -9228,0 -9229,0 -9230,0 -9231,0 -9232,0 -9233,0 -9234,0 -9235,0 -9236,0 -9237,0 -9238,0 -9239,0 -9240,0 -9241,0 -9242,0 -9243,0 -9244,0 -9245,0 -9246,0 -9247,0 -9248,0 -9249,0 -9250,0 -9251,0 -9252,0 -9253,0 -9254,0 -9255,0 -9256,0 -9257,0 -9258,0 -9259,0 -9260,0 -9261,0 -9262,0 -9263,0 -9264,0 -9265,0 -9266,0 -9267,0 -9268,0 -9269,0 -9270,0 -9271,0 -9272,0 -9273,0 -9274,0 -9275,0 -9276,0 -9277,0 -9278,0 -9279,0 -9280,0 -9281,0 -9282,0 -9283,0 -9284,0 -9285,0 -9286,0 -9287,0 -9288,0 -9289,0 -9290,0 -9291,0 -9292,0 -9293,0 -9294,0 -9295,0 -9296,0 -9297,0 -9298,0 -9299,0 -9300,0 -9301,0 -9302,0 -9303,0 -9304,0 -9305,0 -9306,0 -9307,0 -9308,0 -9309,0 -9310,0 -9311,0 -9312,0 -9313,0 -9314,0 -9315,0 -9316,0 -9317,0 -9318,0 -9319,0 -9320,0 -9321,0 -9322,0 -9323,0 -9324,0 -9325,0 -9326,0 -9327,0 -9328,0 -9329,0 -9330,0 -9331,0 -9332,0 -9333,0 -9334,0 -9335,0 -9336,0 -9337,0 -9338,0 -9339,0 -9340,0 -9341,0 -9342,0 -9343,0 -9344,0 -9345,0 -9346,0 -9347,0 -9348,0 -9349,0 -9350,0 -9351,0 -9352,0 -9353,0 -9354,0 -9355,0 -9356,0 -9357,0 -9358,0 -9359,0 -9360,0 -9361,0 -9362,0 -9363,0 -9364,0 -9365,0 -9366,0 -9367,0 -9368,0 -9369,0 -9370,0 -9371,0 -9372,0 -9373,0 -9374,0 -9375,0 -9376,0 -9377,0 -9378,0 -9379,0 -9380,0 -9381,0 -9382,0 -9383,0 -9384,0 -9385,0 -9386,0 -9387,0 -9388,0 -9389,0 -9390,0 -9391,0 -9392,0 -9393,0 -9394,0 -9395,0 -9396,0 -9397,0 -9398,0 -9399,0 -9400,0 -9401,0 -9402,0 -9403,0 -9404,0 -9405,0 -9406,0 -9407,0 -9408,0 -9409,0 -9410,0 -9411,0 -9412,0 -9413,0 -9414,0 -9415,0 -9416,0 -9417,0 -9418,0 -9419,0 -9420,0 -9421,0 -9422,0 -9423,0 -9424,0 -9425,0 -9426,0 -9427,0 -9428,0 -9429,0 -9430,0 -9431,0 -9432,0 -9433,0 -9434,0 -9435,0 -9436,0 -9437,0 -9438,0 -9439,0 -9440,0 -9441,0 -9442,0 -9443,0 -9444,0 -9445,0 -9446,0 -9447,0 -9448,0 -9449,0 -9450,0 -9451,0 -9452,0 -9453,0 -9454,0 -9455,0 -9456,0 -9457,0 -9458,0 -9459,0 -9460,0 -9461,0 -9462,0 -9463,0 -9464,0 -9465,0 -9466,0 -9467,0 -9468,0 -9469,0 -9470,0 -9471,0 -9472,0 -9473,0 -9474,0 -9475,0 -9476,0 -9477,0 -9478,0 -9479,0 -9480,0 -9481,0 -9482,0 -9483,0 -9484,0 -9485,0 -9486,0 -9487,0 -9488,0 -9489,0 -9490,0 -9491,0 -9492,0 -9493,0 -9494,0 -9495,0 -9496,0 -9497,0 -9498,0 -9499,0 -9500,0 -9501,0 -9502,0 -9503,0 -9504,0 -9505,0 -9506,0 -9507,0 -9508,0 -9509,0 -9510,0 -9511,0 -9512,0 -9513,0 -9514,0 -9515,0 -9516,0 -9517,0 -9518,0 -9519,0 -9520,0 -9521,0 -9522,0 -9523,0 -9524,0 -9525,0 -9526,0 -9527,0 -9528,0 -9529,0 -9530,0 -9531,0 -9532,0 -9533,0 -9534,0 -9535,0 -9536,0 -9537,0 -9538,0 -9539,0 -9540,0 -9541,0 -9542,0 -9543,0 -9544,0 -9545,0 -9546,0 -9547,0 -9548,0 -9549,0 -9550,0 -9551,0 -9552,0 -9553,0 -9554,0 -9555,0 -9556,0 -9557,0 -9558,0 -9559,0 -9560,0 -9561,0 -9562,0 -9563,0 -9564,0 -9565,0 -9566,0 -9567,0 -9568,0 -9569,0 -9570,0 -9571,0 -9572,0 -9573,0 -9574,0 -9575,0 -9576,0 -9577,0 -9578,0 -9579,0 -9580,0 -9581,0 -9582,0 -9583,0 -9584,0 -9585,0 -9586,0 -9587,0 -9588,0 -9589,0 -9590,0 -9591,0 -9592,0 -9593,0 -9594,0 -9595,0 -9596,0 -9597,0 -9598,0 -9599,0 -9600,0 -9601,0 -9602,0 -9603,0 -9604,0 -9605,0 -9606,0 -9607,0 -9608,0 -9609,0 -9610,0 -9611,0 -9612,0 -9613,0 -9614,0 -9615,0 -9616,0 -9617,0 -9618,0 -9619,0 -9620,0 -9621,0 -9622,0 -9623,0 -9624,0 -9625,0 -9626,0 -9627,0 -9628,0 -9629,0 -9630,0 -9631,0 -9632,0 -9633,0 -9634,0 -9635,0 -9636,0 -9637,0 -9638,0 -9639,0 -9640,0 -9641,0 -9642,0 -9643,0 -9644,0 -9645,0 -9646,0 -9647,0 -9648,0 -9649,0 -9650,0 -9651,0 -9652,0 -9653,0 -9654,0 -9655,0 -9656,0 -9657,0 -9658,0 -9659,0 -9660,0 -9661,0 -9662,0 -9663,0 -9664,0 -9665,0 -9666,0 -9667,0 -9668,0 -9669,0 -9670,0 -9671,0 -9672,0 -9673,0 -9674,0 -9675,0 -9676,0 -9677,0 -9678,0 -9679,0 -9680,0 -9681,0 -9682,0 -9683,0 -9684,0 -9685,0 -9686,0 -9687,0 -9688,0 -9689,0 -9690,0 -9691,0 -9692,0 -9693,0 -9694,0 -9695,0 -9696,0 -9697,0 -9698,0 -9699,0 -9700,0 -9701,0 -9702,0 -9703,0 -9704,0 -9705,0 -9706,0 -9707,0 -9708,0 -9709,0 -9710,0 -9711,0 -9712,0 -9713,0 -9714,0 -9715,0 -9716,0 -9717,0 -9718,0 -9719,0 -9720,0 -9721,0 -9722,0 -9723,0 -9724,0 -9725,0 -9726,0 -9727,0 -9728,0 -9729,0 -9730,0 -9731,0 -9732,0 -9733,0 -9734,0 -9735,0 -9736,0 -9737,0 -9738,0 -9739,0 -9740,0 -9741,0 -9742,0 -9743,0 -9744,0 -9745,0 -9746,0 -9747,0 -9748,0 -9749,0 -9750,0 -9751,0 -9752,0 -9753,0 -9754,0 -9755,0 -9756,0 -9757,0 -9758,0 -9759,0 -9760,0 -9761,0 -9762,0 -9763,0 -9764,0 -9765,0 -9766,0 -9767,0 -9768,0 -9769,0 -9770,0 -9771,0 -9772,0 -9773,0 -9774,0 -9775,0 -9776,0 -9777,0 -9778,0 -9779,0 -9780,0 -9781,0 -9782,0 -9783,0 -9784,0 -9785,0 -9786,0 -9787,0 -9788,0 -9789,0 -9790,0 -9791,0 -9792,0 -9793,0 -9794,0 -9795,0 -9796,0 -9797,0 -9798,0 -9799,0 -9800,0 -9801,0 -9802,0 -9803,0 -9804,0 -9805,0 -9806,0 -9807,0 -9808,0 -9809,0 -9810,0 -9811,0 -9812,0 -9813,0 -9814,0 -9815,0 -9816,0 -9817,0 -9818,0 -9819,0 -9820,0 -9821,0 -9822,0 -9823,0 -9824,0 -9825,0 -9826,0 -9827,0 -9828,0 -9829,0 -9830,0 -9831,0 -9832,0 -9833,0 -9834,0 -9835,0 -9836,0 -9837,0 -9838,0 -9839,0 -9840,0 -9841,0 -9842,0 -9843,0 -9844,0 -9845,0 -9846,0 -9847,0 -9848,0 -9849,0 -9850,0 -9851,0 -9852,0 -9853,0 -9854,0 -9855,0 -9856,0 -9857,0 -9858,0 -9859,0 -9860,0 -9861,0 -9862,0 -9863,0 -9864,0 -9865,0 -9866,0 -9867,0 -9868,0 -9869,0 -9870,0 -9871,0 -9872,0 -9873,0 -9874,0 -9875,0 -9876,0 -9877,0 -9878,0 -9879,0 -9880,0 -9881,0 -9882,0 -9883,0 -9884,0 -9885,0 -9886,0 -9887,0 -9888,0 -9889,0 -9890,0 -9891,0 -9892,0 -9893,0 -9894,0 -9895,0 -9896,0 -9897,0 -9898,0 -9899,0 -9900,0 -9901,0 -9902,0 -9903,0 -9904,0 -9905,0 -9906,0 -9907,0 -9908,0 -9909,0 -9910,0 -9911,0 -9912,0 -9913,0 -9914,0 -9915,0 -9916,0 -9917,0 -9918,0 -9919,0 -9920,0 -9921,0 -9922,0 -9923,0 -9924,0 -9925,0 -9926,0 -9927,0 -9928,0 -9929,0 -9930,0 -9931,0 -9932,0 -9933,0 -9934,0 -9935,0 -9936,0 -9937,0 -9938,0 -9939,0 -9940,0 -9941,0 -9942,0 -9943,0 -9944,0 -9945,0 -9946,0 -9947,0 -9948,0 -9949,0 -9950,0 -9951,0 -9952,0 -9953,0 -9954,0 -9955,0 -9956,0 -9957,0 -9958,0 -9959,0 -9960,0 -9961,0 -9962,0 -9963,0 -9964,0 -9965,0 -9966,0 -9967,0 -9968,0 -9969,0 -9970,0 -9971,0 -9972,0 -9973,0 -9974,0 -9975,0 -9976,0 -9977,0 -9978,0 -9979,0 -9980,0 -9981,0 -9982,0 -9983,0 -9984,0 -9985,0 -9986,0 -9987,0 -9988,0 -9989,0 -9990,0 -9991,0 -9992,0 -9993,0 -9994,0 -9995,0 -9996,0 -9997,0 -9998,0 -9999,0 -10000,0 -10001,0 -10002,0 -10003,0 -10004,0 -10005,0 -10006,0 -10007,0 -10008,0 -10009,0 -10010,0 -10011,0 -10012,0 -10013,0 -10014,0 -10015,0 -10016,0 -10017,0 -10018,0 -10019,0 -10020,0 -10021,0 -10022,0 -10023,0 -10024,0 -10025,0 -10026,0 -10027,0 -10028,0 -10029,0 -10030,0 -10031,0 -10032,0 -10033,0 -10034,0 -10035,0 -10036,0 -10037,0 -10038,0 -10039,0 -10040,0 -10041,0 -10042,0 -10043,0 -10044,0 -10045,0 -10046,0 -10047,0 -10048,0 -10049,0 -10050,0 -10051,0 -10052,0 -10053,0 -10054,0 -10055,0 -10056,0 -10057,0 -10058,0 -10059,0 -10060,0 -10061,0 -10062,0 -10063,0 -10064,0 -10065,0 -10066,0 -10067,0 -10068,0 -10069,0 -10070,0 -10071,0 -10072,0 -10073,0 -10074,0 -10075,0 -10076,0 -10077,0 -10078,0 -10079,0 -10080,0 -10081,0 -10082,0 -10083,0 -10084,0 -10085,0 -10086,0 -10087,0 -10088,0 -10089,0 -10090,0 -10091,0 -10092,0 -10093,0 -10094,0 -10095,0 -10096,0 -10097,0 -10098,0 -10099,0 -10100,0 -10101,0 -10102,0 -10103,0 -10104,0 -10105,0 -10106,0 -10107,0 -10108,0 -10109,0 -10110,0 -10111,0 -10112,0 -10113,0 -10114,0 -10115,0 -10116,0 -10117,0 -10118,0 -10119,0 -10120,0 -10121,0 -10122,0 -10123,0 -10124,0 -10125,0 -10126,0 -10127,0 -10128,0 -10129,0 -10130,0 -10131,0 -10132,0 -10133,0 -10134,0 -10135,0 -10136,0 -10137,0 -10138,0 -10139,0 -10140,0 -10141,0 -10142,0 -10143,0 -10144,0 -10145,0 -10146,0 -10147,0 -10148,0 -10149,0 -10150,0 -10151,0 -10152,0 -10153,0 -10154,0 -10155,0 -10156,0 -10157,0 -10158,0 -10159,0 -10160,0 -10161,0 -10162,0 -10163,0 -10164,0 -10165,0 -10166,0 -10167,0 -10168,0 -10169,0 -10170,0 -10171,0 -10172,0 -10173,0 -10174,0 -10175,0 -10176,0 -10177,0 -10178,0 -10179,0 -10180,0 -10181,0 -10182,0 -10183,0 -10184,0 -10185,0 -10186,0 -10187,0 -10188,0 -10189,0 -10190,0 -10191,0 -10192,0 -10193,0 -10194,0 -10195,0 -10196,0 -10197,0 -10198,0 -10199,0 -10200,0 -10201,0 -10202,0 -10203,0 -10204,0 -10205,0 -10206,0 -10207,0 -10208,0 -10209,0 -10210,0 -10211,0 -10212,0 -10213,0 -10214,0 -10215,0 -10216,0 -10217,0 -10218,0 -10219,0 -10220,0 -10221,0 -10222,0 -10223,0 -10224,0 -10225,0 -10226,0 -10227,0 -10228,0 -10229,0 -10230,0 -10231,0 -10232,0 -10233,0 -10234,0 -10235,0 -10236,0 -10237,0 -10238,0 -10239,0 -10240,0 -10241,0 -10242,0 -10243,0 -10244,0 -10245,0 -10246,0 -10247,0 -10248,0 -10249,0 -10250,0 -10251,0 -10252,0 -10253,0 -10254,0 -10255,0 -10256,0 -10257,0 -10258,0 -10259,0 -10260,0 -10261,0 -10262,0 -10263,0 -10264,0 -10265,0 -10266,0 -10267,0 -10268,0 -10269,0 -10270,0 -10271,0 -10272,0 -10273,0 -10274,0 -10275,0 -10276,0 -10277,0 -10278,0 -10279,0 -10280,0 -10281,0 -10282,0 -10283,0 -10284,0 -10285,0 -10286,0 -10287,0 -10288,0 -10289,0 -10290,0 -10291,0 -10292,0 -10293,0 -10294,0 -10295,0 -10296,0 -10297,0 -10298,0 -10299,0 -10300,0 -10301,0 -10302,0 -10303,0 -10304,0 -10305,0 -10306,0 -10307,0 -10308,0 -10309,0 -10310,0 -10311,0 -10312,0 -10313,0 -10314,0 -10315,0 -10316,0 -10317,0 -10318,0 -10319,0 -10320,0 -10321,0 -10322,0 -10323,0 -10324,0 -10325,0 -10326,0 -10327,0 -10328,0 -10329,0 -10330,0 -10331,0 -10332,0 -10333,0 -10334,0 -10335,0 -10336,0 -10337,0 -10338,0 -10339,0 -10340,0 -10341,0 -10342,0 -10343,0 -10344,0 -10345,0 -10346,0 -10347,0 -10348,0 -10349,0 -10350,0 -10351,0 -10352,0 -10353,0 -10354,0 -10355,0 -10356,0 -10357,0 -10358,0 -10359,0 -10360,0 -10361,0 -10362,0 -10363,0 -10364,0 -10365,0 -10366,0 -10367,0 -10368,0 -10369,0 -10370,0 -10371,0 -10372,0 -10373,0 -10374,0 -10375,0 -10376,0 -10377,0 -10378,0 -10379,0 -10380,0 -10381,0 -10382,0 -10383,0 -10384,0 -10385,0 -10386,0 -10387,0 -10388,0 -10389,0 -10390,0 -10391,0 -10392,0 -10393,0 -10394,0 -10395,0 -10396,0 -10397,0 -10398,0 -10399,0 -10400,0 -10401,0 -10402,0 -10403,0 -10404,0 -10405,0 -10406,0 -10407,0 -10408,0 -10409,0 -10410,0 -10411,0 -10412,0 -10413,0 -10414,0 -10415,0 -10416,0 -10417,0 -10418,0 -10419,0 -10420,0 -10421,0 -10422,0 -10423,0 -10424,0 -10425,0 -10426,0 -10427,0 -10428,0 -10429,0 -10430,0 -10431,0 -10432,0 -10433,0 -10434,0 -10435,0 -10436,0 -10437,0 -10438,0 -10439,0 -10440,0 -10441,0 -10442,0 -10443,0 -10444,0 -10445,0 -10446,0 -10447,0 -10448,0 -10449,0 -10450,0 -10451,0 -10452,0 -10453,0 -10454,0 -10455,0 -10456,0 -10457,0 -10458,0 -10459,0 -10460,0 -10461,0 -10462,0 -10463,0 -10464,0 -10465,0 -10466,0 -10467,0 -10468,0 -10469,0 -10470,0 -10471,0 -10472,0 -10473,0 -10474,0 -10475,0 -10476,0 -10477,0 -10478,0 -10479,0 -10480,0 -10481,0 -10482,0 -10483,0 -10484,0 -10485,0 -10486,0 -10487,0 -10488,0 -10489,0 -10490,0 -10491,0 -10492,0 -10493,0 -10494,0 -10495,0 -10496,0 -10497,0 -10498,0 -10499,0 -10500,0 -10501,0 -10502,0 -10503,0 -10504,0 -10505,0 -10506,0 -10507,0 -10508,0 -10509,0 -10510,0 -10511,0 -10512,0 -10513,0 -10514,0 -10515,0 -10516,0 -10517,0 -10518,0 -10519,0 -10520,0 -10521,0 -10522,0 -10523,0 -10524,0 -10525,0 -10526,0 -10527,0 -10528,0 -10529,0 -10530,0 -10531,0 -10532,0 -10533,0 -10534,0 -10535,0 -10536,0 -10537,0 -10538,0 -10539,0 -10540,0 -10541,0 -10542,0 -10543,0 -10544,0 -10545,0 -10546,0 -10547,0 -10548,0 -10549,0 -10550,0 -10551,0 -10552,0 -10553,0 -10554,0 -10555,0 -10556,0 -10557,0 -10558,0 -10559,0 -10560,0 -10561,0 -10562,0 -10563,0 -10564,0 -10565,0 -10566,0 -10567,0 -10568,0 -10569,0 -10570,0 -10571,0 -10572,0 -10573,0 -10574,0 -10575,0 -10576,0 -10577,0 -10578,0 -10579,0 -10580,0 -10581,0 -10582,0 -10583,0 -10584,0 -10585,0 -10586,0 -10587,0 -10588,0 -10589,0 -10590,0 -10591,0 -10592,0 -10593,0 -10594,0 -10595,0 -10596,0 -10597,0 -10598,0 -10599,0 -10600,0 -10601,0 -10602,0 -10603,0 -10604,0 -10605,0 -10606,0 -10607,0 -10608,0 -10609,0 -10610,0 -10611,0 -10612,0 -10613,0 -10614,0 -10615,0 -10616,0 -10617,0 -10618,0 -10619,0 -10620,0 -10621,0 -10622,0 -10623,0 -10624,0 -10625,0 -10626,0 -10627,0 -10628,0 -10629,0 -10630,0 -10631,0 -10632,0 -10633,0 -10634,0 -10635,0 -10636,0 -10637,0 -10638,0 -10639,0 -10640,0 -10641,0 -10642,0 -10643,0 -10644,0 -10645,0 -10646,0 -10647,0 -10648,0 -10649,0 -10650,0 -10651,0 -10652,0 -10653,0 -10654,0 -10655,0 -10656,0 -10657,0 -10658,0 -10659,0 -10660,0 -10661,0 -10662,0 -10663,0 -10664,0 -10665,0 -10666,0 -10667,0 -10668,0 -10669,0 -10670,0 -10671,0 -10672,0 -10673,0 -10674,0 -10675,0 -10676,0 -10677,0 -10678,0 -10679,0 -10680,0 -10681,0 -10682,0 -10683,0 -10684,0 -10685,0 -10686,0 -10687,0 -10688,0 -10689,0 -10690,0 -10691,0 -10692,0 -10693,0 -10694,0 -10695,0 -10696,0 -10697,0 -10698,0 -10699,0 -10700,0 -10701,0 -10702,0 -10703,0 -10704,0 -10705,0 -10706,0 -10707,0 -10708,0 -10709,0 -10710,0 -10711,0 -10712,0 -10713,0 -10714,0 -10715,0 -10716,0 -10717,0 -10718,0 -10719,0 -10720,0 -10721,0 -10722,0 -10723,0 -10724,0 -10725,0 -10726,0 -10727,0 -10728,0 -10729,0 -10730,0 -10731,0 -10732,0 -10733,0 -10734,0 -10735,0 -10736,0 -10737,0 -10738,0 -10739,0 -10740,0 -10741,0 -10742,0 -10743,0 -10744,0 -10745,0 -10746,0 -10747,0 -10748,0 -10749,0 -10750,0 -10751,0 -10752,0 -10753,0 -10754,0 -10755,0 -10756,0 -10757,0 -10758,0 -10759,0 -10760,0 -10761,0 -10762,0 -10763,0 -10764,0 -10765,0 -10766,0 -10767,0 -10768,0 -10769,0 -10770,0 -10771,0 -10772,0 -10773,0 -10774,0 -10775,0 -10776,0 -10777,0 -10778,0 -10779,0 -10780,0 -10781,0 -10782,0 -10783,0 -10784,0 -10785,0 -10786,0 -10787,0 -10788,0 -10789,0 -10790,0 -10791,0 -10792,0 -10793,0 -10794,0 -10795,0 -10796,0 -10797,0 -10798,0 -10799,0 -10800,0 -10801,0 -10802,0 -10803,0 -10804,0 -10805,0 -10806,0 -10807,0 -10808,0 -10809,0 -10810,0 -10811,0 -10812,0 -10813,0 -10814,0 -10815,0 -10816,0 -10817,0 -10818,0 -10819,0 -10820,0 -10821,0 -10822,0 -10823,0 -10824,0 -10825,0 -10826,0 -10827,0 -10828,0 -10829,0 -10830,0 -10831,0 -10832,0 -10833,0 -10834,0 -10835,0 -10836,0 -10837,0 -10838,0 -10839,0 -10840,0 -10841,0 -10842,0 -10843,0 -10844,0 -10845,0 -10846,0 -10847,0 -10848,0 -10849,0 -10850,0 -10851,0 -10852,0 -10853,0 -10854,0 -10855,0 -10856,0 -10857,0 -10858,0 -10859,0 -10860,0 -10861,0 -10862,0 -10863,0 -10864,0 -10865,0 -10866,0 -10867,0 -10868,0 -10869,0 -10870,0 -10871,0 -10872,0 -10873,0 -10874,0 -10875,0 -10876,0 -10877,0 -10878,0 -10879,0 -10880,0 -10881,0 -10882,0 -10883,0 -10884,0 -10885,0 -10886,0 -10887,0 -10888,0 -10889,0 -10890,0 -10891,0 -10892,0 -10893,0 -10894,0 -10895,0 -10896,0 -10897,0 -10898,0 -10899,0 -10900,0 -10901,0 -10902,0 -10903,0 -10904,0 -10905,0 -10906,0 -10907,0 -10908,0 -10909,0 -10910,0 -10911,0 -10912,0 -10913,0 -10914,0 -10915,0 -10916,0 -10917,0 -10918,0 -10919,0 -10920,0 -10921,0 -10922,0 -10923,0 -10924,0 -10925,0 -10926,0 -10927,0 -10928,0 -10929,0 -10930,0 -10931,0 -10932,0 -10933,0 -10934,0 -10935,0 -10936,0 -10937,0 -10938,0 -10939,0 -10940,0 -10941,0 -10942,0 -10943,0 -10944,0 -10945,0 -10946,0 -10947,0 -10948,0 -10949,0 -10950,0 -10951,0 -10952,0 -10953,0 -10954,0 -10955,0 -10956,0 -10957,0 -10958,0 -10959,0 -10960,0 -10961,0 -10962,0 -10963,0 -10964,0 -10965,0 -10966,0 -10967,0 -10968,0 -10969,0 -10970,0 -10971,0 -10972,0 -10973,0 -10974,0 -10975,0 -10976,0 -10977,0 -10978,0 -10979,0 -10980,0 -10981,0 -10982,0 -10983,0 -10984,0 -10985,0 -10986,0 -10987,0 -10988,0 -10989,0 -10990,0 -10991,0 -10992,0 -10993,0 -10994,0 -10995,0 -10996,0 -10997,0 -10998,0 -10999,0 -11000,0 -11001,0 -11002,0 -11003,0 -11004,0 -11005,0 -11006,0 -11007,0 -11008,0 -11009,0 -11010,0 -11011,0 -11012,0 -11013,0 -11014,0 -11015,0 -11016,0 -11017,0 -11018,0 -11019,0 -11020,0 -11021,0 -11022,0 -11023,0 -11024,0 -11025,0 -11026,0 -11027,0 -11028,0 -11029,0 -11030,0 -11031,0 -11032,0 -11033,0 -11034,0 -11035,0 -11036,0 -11037,0 -11038,0 -11039,0 -11040,0 -11041,0 -11042,0 -11043,0 -11044,0 -11045,0 -11046,0 -11047,0 -11048,0 -11049,0 -11050,0 -11051,0 -11052,0 -11053,0 -11054,0 -11055,0 -11056,0 -11057,0 -11058,0 -11059,0 -11060,0 -11061,0 -11062,0 -11063,0 -11064,0 -11065,0 -11066,0 -11067,0 -11068,0 -11069,0 -11070,0 -11071,0 -11072,0 -11073,0 -11074,0 -11075,0 -11076,0 -11077,0 -11078,0 -11079,0 -11080,0 -11081,0 -11082,0 -11083,0 -11084,0 -11085,0 -11086,0 -11087,0 -11088,0 -11089,0 -11090,0 -11091,0 -11092,0 -11093,0 -11094,0 -11095,0 -11096,0 -11097,0 -11098,0 -11099,0 -11100,0 -11101,0 -11102,0 -11103,0 -11104,0 -11105,0 -11106,0 -11107,0 -11108,0 -11109,0 -11110,0 -11111,0 -11112,0 -11113,0 -11114,0 -11115,0 -11116,0 -11117,0 -11118,0 -11119,0 -11120,0 -11121,0 -11122,0 -11123,0 -11124,0 -11125,0 -11126,0 -11127,0 -11128,0 -11129,0 -11130,0 -11131,0 -11132,0 -11133,0 -11134,0 -11135,0 -11136,0 -11137,0 -11138,0 -11139,0 -11140,0 -11141,0 -11142,0 -11143,0 -11144,0 -11145,0 -11146,0 -11147,0 -11148,0 -11149,0 -11150,0 -11151,0 -11152,0 -11153,0 -11154,0 -11155,0 -11156,0 -11157,0 -11158,0 -11159,0 -11160,0 -11161,0 -11162,0 -11163,0 -11164,0 -11165,0 -11166,0 -11167,0 -11168,0 -11169,0 -11170,0 -11171,0 -11172,0 -11173,0 -11174,0 -11175,0 -11176,0 -11177,0 -11178,0 -11179,0 -11180,0 -11181,0 -11182,0 -11183,0 -11184,0 -11185,0 -11186,0 -11187,0 -11188,0 -11189,0 -11190,0 -11191,0 -11192,0 -11193,0 -11194,0 -11195,0 -11196,0 -11197,0 -11198,0 -11199,0 -11200,0 -11201,0 -11202,0 -11203,0 -11204,0 -11205,0 -11206,0 -11207,0 -11208,0 -11209,0 -11210,0 -11211,0 -11212,0 -11213,0 -11214,0 -11215,0 -11216,0 -11217,0 -11218,0 -11219,0 -11220,0 -11221,0 -11222,0 -11223,0 -11224,0 -11225,0 -11226,0 -11227,0 -11228,0 -11229,0 -11230,0 -11231,0 -11232,0 -11233,0 -11234,0 -11235,0 -11236,0 -11237,0 -11238,0 -11239,0 -11240,0 -11241,0 -11242,0 -11243,0 -11244,0 -11245,0 -11246,0 -11247,0 -11248,0 -11249,0 -11250,0 -11251,0 -11252,0 -11253,0 -11254,0 -11255,0 -11256,0 -11257,0 -11258,0 -11259,0 -11260,0 -11261,0 -11262,0 -11263,0 -11264,0 -11265,0 -11266,0 -11267,0 -11268,0 -11269,0 -11270,0 -11271,0 -11272,0 -11273,0 -11274,0 -11275,0 -11276,0 -11277,0 -11278,0 -11279,0 -11280,0 -11281,0 -11282,0 -11283,0 -11284,0 -11285,0 -11286,0 -11287,0 -11288,0 -11289,0 -11290,0 -11291,0 -11292,0 -11293,0 -11294,0 -11295,0 -11296,0 -11297,0 -11298,0 -11299,0 -11300,0 -11301,0 -11302,0 -11303,0 -11304,0 -11305,0 -11306,0 -11307,0 -11308,0 -11309,0 -11310,0 -11311,0 -11312,0 -11313,0 -11314,0 -11315,0 -11316,0 -11317,0 -11318,0 -11319,0 -11320,0 -11321,0 -11322,0 -11323,0 -11324,0 -11325,0 -11326,0 -11327,0 -11328,0 -11329,0 -11330,0 -11331,0 -11332,0 -11333,0 -11334,0 -11335,0 -11336,0 -11337,0 -11338,0 -11339,0 -11340,0 -11341,0 -11342,0 -11343,0 -11344,0 -11345,0 -11346,0 -11347,0 -11348,0 -11349,0 -11350,0 -11351,0 -11352,0 -11353,0 -11354,0 -11355,0 -11356,0 -11357,0 -11358,0 -11359,0 -11360,0 -11361,0 -11362,0 -11363,0 -11364,0 -11365,0 -11366,0 -11367,0 -11368,0 -11369,0 -11370,0 -11371,0 -11372,0 -11373,0 -11374,0 -11375,0 -11376,0 -11377,0 -11378,0 -11379,0 -11380,0 -11381,0 -11382,0 -11383,0 -11384,0 -11385,0 -11386,0 -11387,0 -11388,0 -11389,0 -11390,0 -11391,0 -11392,0 -11393,0 -11394,0 -11395,0 -11396,0 -11397,0 -11398,0 -11399,0 -11400,0 -11401,0 -11402,0 -11403,0 -11404,0 -11405,0 -11406,0 -11407,0 -11408,0 -11409,0 -11410,0 -11411,0 -11412,0 -11413,0 -11414,0 -11415,0 -11416,0 -11417,0 -11418,0 -11419,0 -11420,0 -11421,0 -11422,0 -11423,0 -11424,0 -11425,0 -11426,0 -11427,0 -11428,0 -11429,0 -11430,0 -11431,0 -11432,0 -11433,0 -11434,0 -11435,0 -11436,0 -11437,0 -11438,0 -11439,0 -11440,0 -11441,0 -11442,0 -11443,0 -11444,0 -11445,0 -11446,0 -11447,0 -11448,0 -11449,0 -11450,0 -11451,0 -11452,0 -11453,0 -11454,0 -11455,0 -11456,0 -11457,0 -11458,0 -11459,0 -11460,0 -11461,0 -11462,0 -11463,0 -11464,0 -11465,0 -11466,0 -11467,0 -11468,0 -11469,0 -11470,0 -11471,0 -11472,0 -11473,0 -11474,0 -11475,0 -11476,0 -11477,0 -11478,0 -11479,0 -11480,0 -11481,0 -11482,0 -11483,0 -11484,0 -11485,0 -11486,0 -11487,0 -11488,0 -11489,0 -11490,0 -11491,0 -11492,0 -11493,0 -11494,0 -11495,0 -11496,0 -11497,0 -11498,0 -11499,0 -11500,0 -11501,0 -11502,0 -11503,0 -11504,0 -11505,0 -11506,0 -11507,0 -11508,0 -11509,0 -11510,0 -11511,0 -11512,0 -11513,0 -11514,0 -11515,0 -11516,0 -11517,0 -11518,0 -11519,0 -11520,0 -11521,0 -11522,0 -11523,0 -11524,0 -11525,0 -11526,0 -11527,0 -11528,0 -11529,0 -11530,0 -11531,0 -11532,0 -11533,0 -11534,0 -11535,0 -11536,0 -11537,0 -11538,0 -11539,0 -11540,0 -11541,0 -11542,0 -11543,0 -11544,0 -11545,0 -11546,0 -11547,0 -11548,0 -11549,0 -11550,0 -11551,0 -11552,0 -11553,0 -11554,0 -11555,0 -11556,0 -11557,0 -11558,0 -11559,0 -11560,0 -11561,0 -11562,0 -11563,0 -11564,0 -11565,0 -11566,0 -11567,0 -11568,0 -11569,0 -11570,0 -11571,0 -11572,0 -11573,0 -11574,0 -11575,0 -11576,0 -11577,0 -11578,0 -11579,0 -11580,0 -11581,0 -11582,0 -11583,0 -11584,0 -11585,0 -11586,0 -11587,0 -11588,0 -11589,0 -11590,0 -11591,0 -11592,0 -11593,0 -11594,0 -11595,0 -11596,0 -11597,0 -11598,0 -11599,0 -11600,0 -11601,0 -11602,0 -11603,0 -11604,0 -11605,0 -11606,0 -11607,0 -11608,0 -11609,0 -11610,0 -11611,0 -11612,0 -11613,0 -11614,0 -11615,0 -11616,0 -11617,0 -11618,0 -11619,0 -11620,0 -11621,0 -11622,0 -11623,0 -11624,0 -11625,0 -11626,0 -11627,0 -11628,0 -11629,0 -11630,0 -11631,0 -11632,0 -11633,0 -11634,0 -11635,0 -11636,0 -11637,0 -11638,0 -11639,0 -11640,0 -11641,0 -11642,0 -11643,0 -11644,0 -11645,0 -11646,0 -11647,0 -11648,0 -11649,0 -11650,0 -11651,0 -11652,0 -11653,0 -11654,0 -11655,0 -11656,0 -11657,0 -11658,0 -11659,0 -11660,0 -11661,0 -11662,0 -11663,0 -11664,0 -11665,0 -11666,0 -11667,0 -11668,0 -11669,0 -11670,0 -11671,0 -11672,0 -11673,0 -11674,0 -11675,0 -11676,0 -11677,0 -11678,0 -11679,0 -11680,0 -11681,0 -11682,0 -11683,0 -11684,0 -11685,0 -11686,0 -11687,0 -11688,0 -11689,0 -11690,0 -11691,0 -11692,0 -11693,0 -11694,0 -11695,0 -11696,0 -11697,0 -11698,0 -11699,0 -11700,0 -11701,0 -11702,0 -11703,0 -11704,0 -11705,0 -11706,0 -11707,0 -11708,0 -11709,0 -11710,0 -11711,0 -11712,0 -11713,0 -11714,0 -11715,0 -11716,0 -11717,0 -11718,0 -11719,0 -11720,0 -11721,0 -11722,0 -11723,0 -11724,0 -11725,0 -11726,0 -11727,0 -11728,0 -11729,0 -11730,0 -11731,0 -11732,0 -11733,0 -11734,0 -11735,0 -11736,0 -11737,0 -11738,0 -11739,0 -11740,0 -11741,0 -11742,0 -11743,0 -11744,0 -11745,0 -11746,0 -11747,0 -11748,0 -11749,0 -11750,0 -11751,0 -11752,0 -11753,0 -11754,0 -11755,0 -11756,0 -11757,0 -11758,0 -11759,0 -11760,0 -11761,0 -11762,0 -11763,0 -11764,0 -11765,0 -11766,0 -11767,0 -11768,0 -11769,0 -11770,0 -11771,0 -11772,0 -11773,0 -11774,0 -11775,0 -11776,0 -11777,0 -11778,0 -11779,0 -11780,0 -11781,0 -11782,0 -11783,0 -11784,0 -11785,0 -11786,0 -11787,0 -11788,0 -11789,0 -11790,0 -11791,0 -11792,0 -11793,0 -11794,0 -11795,0 -11796,0 -11797,0 -11798,0 -11799,0 -11800,0 -11801,0 -11802,0 -11803,0 -11804,0 -11805,0 -11806,0 -11807,0 -11808,0 -11809,0 -11810,0 -11811,0 -11812,0 -11813,0 -11814,0 -11815,0 -11816,0 -11817,0 -11818,0 -11819,0 -11820,0 -11821,0 -11822,0 -11823,0 -11824,0 -11825,0 -11826,0 -11827,0 -11828,0 -11829,0 -11830,0 -11831,0 -11832,0 -11833,0 -11834,0 -11835,0 -11836,0 -11837,0 -11838,0 -11839,0 -11840,0 -11841,0 -11842,0 -11843,0 -11844,0 -11845,0 -11846,0 -11847,0 -11848,0 -11849,0 -11850,0 -11851,0 -11852,0 -11853,0 -11854,0 -11855,0 -11856,0 -11857,0 -11858,0 -11859,0 -11860,0 -11861,0 -11862,0 -11863,0 -11864,0 -11865,0 -11866,0 -11867,0 -11868,0 -11869,0 -11870,0 -11871,0 -11872,0 -11873,0 -11874,0 -11875,0 -11876,0 -11877,0 -11878,0 -11879,0 -11880,0 -11881,0 -11882,0 -11883,0 -11884,0 -11885,0 -11886,0 -11887,0 -11888,0 -11889,0 -11890,0 -11891,0 -11892,0 -11893,0 -11894,0 -11895,0 -11896,0 -11897,0 -11898,0 -11899,0 -11900,0 -11901,0 -11902,0 -11903,0 -11904,0 -11905,0 -11906,0 -11907,0 -11908,0 -11909,0 -11910,0 -11911,0 -11912,0 -11913,0 -11914,0 -11915,0 -11916,0 -11917,0 -11918,0 -11919,0 -11920,0 -11921,0 -11922,0 -11923,0 -11924,0 -11925,0 -11926,0 -11927,0 -11928,0 -11929,0 -11930,0 -11931,0 -11932,0 -11933,0 -11934,0 -11935,0 -11936,0 -11937,0 -11938,0 -11939,0 -11940,0 -11941,0 -11942,0 -11943,0 -11944,0 -11945,0 -11946,0 -11947,0 -11948,0 -11949,0 -11950,0 -11951,0 -11952,0 -11953,0 -11954,0 -11955,0 -11956,0 -11957,0 -11958,0 -11959,0 -11960,0 -11961,0 -11962,0 -11963,0 -11964,0 -11965,0 -11966,0 -11967,0 -11968,0 -11969,0 -11970,0 -11971,0 -11972,0 -11973,0 -11974,0 -11975,0 -11976,0 -11977,0 -11978,0 -11979,0 -11980,0 -11981,0 -11982,0 -11983,0 -11984,0 -11985,0 -11986,0 -11987,0 -11988,0 -11989,0 -11990,0 -11991,0 -11992,0 -11993,0 -11994,0 -11995,0 -11996,0 -11997,0 -11998,0 -11999,0 -12000,0 -12001,0 -12002,0 -12003,0 -12004,0 -12005,0 -12006,0 -12007,0 -12008,0 -12009,0 -12010,0 -12011,0 -12012,0 -12013,0 -12014,0 -12015,0 -12016,0 -12017,0 -12018,0 -12019,0 -12020,0 -12021,0 -12022,0 -12023,0 -12024,0 -12025,0 -12026,0 -12027,0 -12028,0 -12029,0 -12030,0 -12031,0 -12032,0 -12033,0 -12034,0 -12035,0 -12036,0 -12037,0 -12038,0 -12039,0 -12040,0 -12041,0 -12042,0 -12043,0 -12044,0 -12045,0 -12046,0 -12047,0 -12048,0 -12049,0 -12050,0 -12051,0 -12052,0 -12053,0 -12054,0 -12055,0 -12056,0 -12057,0 -12058,0 -12059,0 -12060,0 -12061,0 -12062,0 -12063,0 -12064,0 -12065,0 -12066,0 -12067,0 -12068,0 -12069,0 -12070,0 -12071,0 -12072,0 -12073,0 -12074,0 -12075,0 -12076,0 -12077,0 -12078,0 -12079,0 -12080,0 -12081,0 -12082,0 -12083,0 -12084,0 -12085,0 -12086,0 -12087,0 -12088,0 -12089,0 -12090,0 -12091,0 -12092,0 -12093,0 -12094,0 -12095,0 -12096,0 -12097,0 -12098,0 -12099,0 -12100,0 -12101,0 -12102,0 -12103,0 -12104,0 -12105,0 -12106,0 -12107,0 -12108,0 -12109,0 -12110,0 -12111,0 -12112,0 -12113,0 -12114,0 -12115,0 -12116,0 -12117,0 -12118,0 -12119,0 -12120,0 -12121,0 -12122,0 -12123,0 -12124,0 -12125,0 -12126,0 -12127,0 -12128,0 -12129,0 -12130,0 -12131,0 -12132,0 -12133,0 -12134,0 -12135,0 -12136,0 -12137,0 -12138,0 -12139,0 -12140,0 -12141,0 -12142,0 -12143,0 -12144,0 -12145,0 -12146,0 -12147,0 -12148,0 -12149,0 -12150,0 -12151,0 -12152,0 -12153,0 -12154,0 -12155,0 -12156,0 -12157,0 -12158,0 -12159,0 -12160,0 -12161,0 -12162,0 -12163,0 -12164,0 -12165,0 -12166,0 -12167,0 -12168,0 -12169,0 -12170,0 -12171,0 -12172,0 -12173,0 -12174,0 -12175,0 -12176,0 -12177,0 -12178,0 -12179,0 -12180,0 -12181,0 -12182,0 -12183,0 -12184,0 -12185,0 -12186,0 -12187,0 -12188,0 -12189,0 -12190,0 -12191,0 -12192,0 -12193,0 -12194,0 -12195,0 -12196,0 -12197,0 -12198,0 -12199,0 -12200,0 -12201,0 -12202,0 -12203,0 -12204,0 -12205,0 -12206,0 -12207,0 -12208,0 -12209,0 -12210,0 -12211,0 -12212,0 -12213,0 -12214,0 -12215,0 -12216,0 -12217,0 -12218,0 -12219,0 -12220,0 -12221,0 -12222,0 -12223,0 -12224,0 -12225,0 -12226,0 -12227,0 -12228,0 -12229,0 -12230,0 -12231,0 -12232,0 -12233,0 -12234,0 -12235,0 -12236,0 -12237,0 -12238,0 -12239,0 -12240,0 -12241,0 -12242,0 -12243,0 -12244,0 -12245,0 -12246,0 -12247,0 -12248,0 -12249,0 -12250,0 -12251,0 -12252,0 -12253,0 -12254,0 -12255,0 -12256,0 -12257,0 -12258,0 -12259,0 -12260,0 -12261,0 -12262,0 -12263,0 -12264,0 -12265,0 -12266,0 -12267,0 -12268,0 -12269,0 -12270,0 -12271,0 -12272,0 -12273,0 -12274,0 -12275,0 -12276,0 -12277,0 -12278,0 -12279,0 -12280,0 -12281,0 -12282,0 -12283,0 -12284,0 -12285,0 -12286,0 -12287,0 -12288,0 -12289,0 -12290,0 -12291,0 -12292,0 -12293,0 -12294,0 -12295,0 -12296,0 -12297,0 -12298,0 -12299,0 -12300,0 -12301,0 -12302,0 -12303,0 -12304,0 -12305,0 -12306,0 -12307,0 -12308,0 -12309,0 -12310,0 -12311,0 -12312,0 -12313,0 -12314,0 -12315,0 -12316,0 -12317,0 -12318,0 -12319,0 -12320,0 -12321,0 -12322,0 -12323,0 -12324,0 -12325,0 -12326,0 -12327,0 -12328,0 -12329,0 -12330,0 -12331,0 -12332,0 -12333,0 -12334,0 -12335,0 -12336,0 -12337,0 -12338,0 -12339,0 -12340,0 -12341,0 -12342,0 -12343,0 -12344,0 -12345,0 -12346,0 -12347,0 -12348,0 -12349,0 -12350,0 -12351,0 -12352,0 -12353,0 -12354,0 -12355,0 -12356,0 -12357,0 -12358,0 -12359,0 -12360,0 -12361,0 -12362,0 -12363,0 -12364,0 -12365,0 -12366,0 -12367,0 -12368,0 -12369,0 -12370,0 -12371,0 -12372,0 -12373,0 -12374,0 -12375,0 -12376,0 -12377,0 -12378,0 -12379,0 -12380,0 -12381,0 -12382,0 -12383,0 -12384,0 -12385,0 -12386,0 -12387,0 -12388,0 -12389,0 -12390,0 -12391,0 -12392,0 -12393,0 -12394,0 -12395,0 -12396,0 -12397,0 -12398,0 -12399,0 -12400,0 -12401,0 -12402,0 -12403,0 -12404,0 -12405,0 -12406,0 -12407,0 -12408,0 -12409,0 -12410,0 -12411,0 -12412,0 -12413,0 -12414,0 -12415,0 -12416,0 -12417,0 -12418,0 -12419,0 -12420,0 -12421,0 -12422,0 -12423,0 -12424,0 -12425,0 -12426,0 -12427,0 -12428,0 -12429,0 -12430,0 -12431,0 -12432,0 -12433,0 -12434,0 -12435,0 -12436,0 -12437,0 -12438,0 -12439,0 -12440,0 -12441,0 -12442,0 -12443,0 -12444,0 -12445,0 -12446,0 -12447,0 -12448,0 -12449,0 -12450,0 -12451,0 -12452,0 -12453,0 -12454,0 -12455,0 -12456,0 -12457,0 -12458,0 -12459,0 -12460,0 -12461,0 -12462,0 -12463,0 -12464,0 -12465,0 -12466,0 -12467,0 -12468,0 -12469,0 -12470,0 -12471,0 -12472,0 -12473,0 -12474,0 -12475,0 -12476,0 -12477,0 -12478,0 -12479,0 -12480,0 -12481,0 -12482,0 -12483,0 -12484,0 -12485,0 -12486,0 -12487,0 -12488,0 -12489,0 -12490,0 -12491,0 -12492,0 -12493,0 -12494,0 -12495,0 -12496,0 -12497,0 -12498,0 -12499,0 -12500,0 -12501,0 -12502,0 -12503,0 -12504,0 -12505,0 -12506,0 -12507,0 -12508,0 -12509,0 -12510,0 -12511,0 -12512,0 -12513,0 -12514,0 -12515,0 -12516,0 -12517,0 -12518,0 -12519,0 -12520,0 -12521,0 -12522,0 -12523,0 -12524,0 -12525,0 -12526,0 -12527,0 -12528,0 -12529,0 -12530,0 -12531,0 -12532,0 -12533,0 -12534,0 -12535,0 -12536,0 -12537,0 -12538,0 -12539,0 -12540,0 -12541,0 -12542,0 -12543,0 -12544,0 -12545,0 -12546,0 -12547,0 -12548,0 -12549,0 -12550,0 -12551,0 -12552,0 -12553,0 -12554,0 -12555,0 -12556,0 -12557,0 -12558,0 -12559,0 -12560,0 -12561,0 -12562,0 -12563,0 -12564,0 -12565,0 -12566,0 -12567,0 -12568,0 -12569,0 -12570,0 -12571,0 -12572,0 -12573,0 -12574,0 -12575,0 -12576,0 -12577,0 -12578,0 -12579,0 -12580,0 -12581,0 -12582,0 -12583,0 -12584,0 -12585,0 -12586,0 -12587,0 -12588,0 -12589,0 -12590,0 -12591,0 -12592,0 -12593,0 -12594,0 -12595,0 -12596,0 -12597,0 -12598,0 -12599,0 -12600,0 -12601,0 -12602,0 -12603,0 -12604,0 -12605,0 -12606,0 -12607,0 -12608,0 -12609,0 -12610,0 -12611,0 -12612,0 -12613,0 -12614,0 -12615,0 -12616,0 -12617,0 -12618,0 -12619,0 -12620,0 -12621,0 -12622,0 -12623,0 -12624,0 -12625,0 -12626,0 -12627,0 -12628,0 -12629,0 -12630,0 -12631,0 -12632,0 -12633,0 -12634,0 -12635,0 -12636,0 -12637,0 -12638,0 -12639,0 -12640,0 -12641,0 -12642,0 -12643,0 -12644,0 -12645,0 -12646,0 -12647,0 -12648,0 -12649,0 -12650,0 -12651,0 -12652,0 -12653,0 -12654,0 -12655,0 -12656,0 -12657,0 -12658,0 -12659,0 -12660,0 -12661,0 -12662,0 -12663,0 -12664,0 -12665,0 -12666,0 -12667,0 -12668,0 -12669,0 -12670,0 -12671,0 -12672,0 -12673,0 -12674,0 -12675,0 -12676,0 -12677,0 -12678,0 -12679,0 -12680,0 -12681,0 -12682,0 -12683,0 -12684,0 -12685,0 -12686,0 -12687,0 -12688,0 -12689,0 -12690,0 -12691,0 -12692,0 -12693,0 -12694,0 -12695,0 -12696,0 -12697,0 -12698,0 -12699,0 -12700,0 -12701,0 -12702,0 -12703,0 -12704,0 -12705,0 -12706,0 -12707,0 -12708,0 -12709,0 -12710,0 -12711,0 -12712,0 -12713,0 -12714,0 -12715,0 -12716,0 -12717,0 -12718,0 -12719,0 -12720,0 -12721,0 -12722,0 -12723,0 -12724,0 -12725,0 -12726,0 -12727,0 -12728,0 -12729,0 -12730,0 -12731,0 -12732,0 -12733,0 -12734,0 -12735,0 -12736,0 -12737,0 -12738,0 -12739,0 -12740,0 -12741,0 -12742,0 -12743,0 -12744,0 -12745,0 -12746,0 -12747,0 -12748,0 -12749,0 -12750,0 -12751,0 -12752,0 -12753,0 -12754,0 -12755,0 -12756,0 -12757,0 -12758,0 -12759,0 -12760,0 -12761,0 -12762,0 -12763,0 -12764,0 -12765,0 -12766,0 -12767,0 -12768,0 -12769,0 -12770,0 -12771,0 -12772,0 -12773,0 -12774,0 -12775,0 -12776,0 -12777,0 -12778,0 -12779,0 -12780,0 -12781,0 -12782,0 -12783,0 -12784,0 -12785,0 -12786,0 -12787,0 -12788,0 -12789,0 -12790,0 -12791,0 -12792,0 -12793,0 -12794,0 -12795,0 -12796,0 -12797,0 -12798,0 -12799,0 -12800,0 -12801,0 -12802,0 -12803,0 -12804,0 -12805,0 -12806,0 -12807,0 -12808,0 -12809,0 -12810,0 -12811,0 -12812,0 -12813,0 -12814,0 -12815,0 -12816,0 -12817,0 -12818,0 -12819,0 -12820,0 -12821,0 -12822,0 -12823,0 -12824,0 -12825,0 -12826,0 -12827,0 -12828,0 -12829,0 -12830,0 -12831,0 -12832,0 -12833,0 -12834,0 -12835,0 -12836,0 -12837,0 -12838,0 -12839,0 -12840,0 -12841,0 -12842,0 -12843,0 -12844,0 -12845,0 -12846,0 -12847,0 -12848,0 -12849,0 -12850,0 -12851,0 -12852,0 -12853,0 -12854,0 -12855,0 -12856,0 -12857,0 -12858,0 -12859,0 -12860,0 -12861,0 -12862,0 -12863,0 -12864,0 -12865,0 -12866,0 -12867,0 -12868,0 -12869,0 -12870,0 -12871,0 -12872,0 -12873,0 -12874,0 -12875,0 -12876,0 -12877,0 -12878,0 -12879,0 -12880,0 -12881,0 -12882,0 -12883,0 -12884,0 -12885,0 -12886,0 -12887,0 -12888,0 -12889,0 -12890,0 -12891,0 -12892,0 -12893,0 -12894,0 -12895,0 -12896,0 -12897,0 -12898,0 -12899,0 -12900,0 -12901,0 -12902,0 -12903,0 -12904,0 -12905,0 -12906,0 -12907,0 -12908,0 -12909,0 -12910,0 -12911,0 -12912,0 -12913,0 -12914,0 -12915,0 -12916,0 -12917,0 -12918,0 -12919,0 -12920,0 -12921,0 -12922,0 -12923,0 -12924,0 -12925,0 -12926,0 -12927,0 -12928,0 -12929,0 -12930,0 -12931,0 -12932,0 -12933,0 -12934,0 -12935,0 -12936,0 -12937,0 -12938,0 -12939,0 -12940,0 -12941,0 -12942,0 -12943,0 -12944,0 -12945,0 -12946,0 -12947,0 -12948,0 -12949,0 -12950,0 -12951,0 -12952,0 -12953,0 -12954,0 -12955,0 -12956,0 -12957,0 -12958,0 -12959,0 -12960,0 -12961,0 -12962,0 -12963,0 -12964,0 -12965,0 -12966,0 -12967,0 -12968,0 -12969,0 -12970,0 -12971,0 -12972,0 -12973,0 -12974,0 -12975,0 -12976,0 -12977,0 -12978,0 -12979,0 -12980,0 -12981,0 -12982,0 -12983,0 -12984,0 -12985,0 -12986,0 -12987,0 -12988,0 -12989,0 -12990,0 -12991,0 -12992,0 -12993,0 -12994,0 -12995,0 -12996,0 -12997,0 -12998,0 -12999,0 -13000,0 -13001,0 -13002,0 -13003,0 -13004,0 -13005,0 -13006,0 -13007,0 -13008,0 -13009,0 -13010,0 -13011,0 -13012,0 -13013,0 -13014,0 -13015,0 -13016,0 -13017,0 -13018,0 -13019,0 -13020,0 -13021,0 -13022,0 -13023,0 -13024,0 -13025,0 -13026,0 -13027,0 -13028,0 -13029,0 -13030,0 -13031,0 -13032,0 -13033,0 -13034,0 -13035,0 -13036,0 -13037,0 -13038,0 -13039,0 -13040,0 -13041,0 -13042,0 -13043,0 -13044,0 -13045,0 -13046,0 -13047,0 -13048,0 -13049,0 -13050,0 -13051,0 -13052,0 -13053,0 -13054,0 -13055,0 -13056,0 -13057,0 -13058,0 -13059,0 -13060,0 -13061,0 -13062,0 -13063,0 -13064,0 -13065,0 -13066,0 -13067,0 -13068,0 -13069,0 -13070,0 -13071,0 -13072,0 -13073,0 -13074,0 -13075,0 -13076,0 -13077,0 -13078,0 -13079,0 -13080,0 -13081,0 -13082,0 -13083,0 -13084,0 -13085,0 -13086,0 -13087,0 -13088,0 -13089,0 -13090,0 -13091,0 -13092,0 -13093,0 -13094,0 -13095,0 -13096,0 -13097,0 -13098,0 -13099,0 -13100,0 -13101,0 -13102,0 -13103,0 -13104,0 -13105,0 -13106,0 -13107,0 -13108,0 -13109,0 -13110,0 -13111,0 -13112,0 -13113,0 -13114,0 -13115,0 -13116,0 -13117,0 -13118,0 -13119,0 -13120,0 -13121,0 -13122,0 -13123,0 -13124,0 -13125,0 -13126,0 -13127,0 -13128,0 -13129,0 -13130,0 -13131,0 -13132,0 -13133,0 -13134,0 -13135,0 -13136,0 -13137,0 -13138,0 -13139,0 -13140,0 -13141,0 -13142,0 -13143,0 -13144,0 -13145,0 -13146,0 -13147,0 -13148,0 -13149,0 -13150,0 -13151,0 -13152,0 -13153,0 -13154,0 -13155,0 -13156,0 -13157,0 -13158,0 -13159,0 -13160,0 -13161,0 -13162,0 -13163,0 -13164,0 -13165,0 -13166,0 -13167,0 -13168,0 -13169,0 -13170,0 -13171,0 -13172,0 -13173,0 -13174,0 -13175,0 -13176,0 -13177,0 -13178,0 -13179,0 -13180,0 -13181,0 -13182,0 -13183,0 -13184,0 -13185,0 -13186,0 -13187,0 -13188,0 -13189,0 -13190,0 -13191,0 -13192,0 -13193,0 -13194,0 -13195,0 -13196,0 -13197,0 -13198,0 -13199,0 -13200,0 -13201,0 -13202,0 -13203,0 -13204,0 -13205,0 -13206,0 -13207,0 -13208,0 -13209,0 -13210,0 -13211,0 -13212,0 -13213,0 -13214,0 -13215,0 -13216,0 -13217,0 -13218,0 -13219,0 -13220,0 -13221,0 -13222,0 -13223,0 -13224,0 -13225,0 -13226,0 -13227,0 -13228,0 -13229,0 -13230,0 -13231,0 -13232,0 -13233,0 -13234,0 -13235,0 -13236,0 -13237,0 -13238,0 -13239,0 -13240,0 -13241,0 -13242,0 -13243,0 -13244,0 -13245,0 -13246,0 -13247,0 -13248,0 -13249,0 -13250,0 -13251,0 -13252,0 -13253,0 -13254,0 -13255,0 -13256,0 -13257,0 -13258,0 -13259,0 -13260,0 -13261,0 -13262,0 -13263,0 -13264,0 -13265,0 -13266,0 -13267,0 -13268,0 -13269,0 -13270,0 -13271,0 -13272,0 -13273,0 -13274,0 -13275,0 -13276,0 -13277,0 -13278,0 -13279,0 -13280,0 -13281,0 -13282,0 -13283,0 -13284,0 -13285,0 -13286,0 -13287,0 -13288,0 -13289,0 -13290,0 -13291,0 -13292,0 -13293,0 -13294,0 -13295,0 -13296,0 -13297,0 -13298,0 -13299,0 -13300,0 -13301,0 -13302,0 -13303,0 -13304,0 -13305,0 -13306,0 -13307,0 -13308,0 -13309,0 -13310,0 -13311,0 -13312,0 -13313,0 -13314,0 -13315,0 -13316,0 -13317,0 -13318,0 -13319,0 -13320,0 -13321,0 -13322,0 -13323,0 -13324,0 -13325,0 -13326,0 -13327,0 -13328,0 -13329,0 -13330,0 -13331,0 -13332,0 -13333,0 -13334,0 -13335,0 -13336,0 -13337,0 -13338,0 -13339,0 -13340,0 -13341,0 -13342,0 -13343,0 -13344,0 -13345,0 -13346,0 -13347,0 -13348,0 -13349,0 -13350,0 -13351,0 -13352,0 -13353,0 -13354,0 -13355,0 -13356,0 -13357,0 -13358,0 -13359,0 -13360,0 -13361,0 -13362,0 -13363,0 -13364,0 -13365,0 -13366,0 -13367,0 -13368,0 -13369,0 -13370,0 -13371,0 -13372,0 -13373,0 -13374,0 -13375,0 -13376,0 -13377,0 -13378,0 -13379,0 -13380,0 -13381,0 -13382,0 -13383,0 -13384,0 -13385,0 -13386,0 -13387,0 -13388,0 -13389,0 -13390,0 -13391,0 -13392,0 -13393,0 -13394,0 -13395,0 -13396,0 -13397,0 -13398,0 -13399,0 -13400,0 -13401,0 -13402,0 -13403,0 -13404,0 -13405,0 -13406,0 -13407,0 -13408,0 -13409,0 -13410,0 -13411,0 -13412,0 -13413,0 -13414,0 -13415,0 -13416,0 -13417,0 -13418,0 -13419,0 -13420,0 -13421,0 -13422,0 -13423,0 -13424,0 -13425,0 -13426,0 -13427,0 -13428,0 -13429,0 -13430,0 -13431,0 -13432,0 -13433,0 -13434,0 -13435,0 -13436,0 -13437,0 -13438,0 -13439,0 -13440,0 -13441,0 -13442,0 -13443,0 -13444,0 -13445,0 -13446,0 -13447,0 -13448,0 -13449,0 -13450,0 -13451,0 -13452,0 -13453,0 -13454,0 -13455,0 -13456,0 -13457,0 -13458,0 -13459,0 -13460,0 -13461,0 -13462,0 -13463,0 -13464,0 -13465,0 -13466,0 -13467,0 -13468,0 -13469,0 -13470,0 -13471,0 -13472,0 -13473,0 -13474,0 -13475,0 -13476,0 -13477,0 -13478,0 -13479,0 -13480,0 -13481,0 -13482,0 -13483,0 -13484,0 -13485,0 -13486,0 -13487,0 -13488,0 -13489,0 -13490,0 -13491,0 -13492,0 -13493,0 -13494,0 -13495,0 -13496,0 -13497,0 -13498,0 -13499,0 -13500,0 -13501,0 -13502,0 -13503,0 -13504,0 -13505,0 -13506,0 -13507,0 -13508,0 -13509,0 -13510,0 -13511,0 -13512,0 -13513,0 -13514,0 -13515,0 -13516,0 -13517,0 -13518,0 -13519,0 -13520,0 -13521,0 -13522,0 -13523,0 -13524,0 -13525,0 -13526,0 -13527,0 -13528,0 -13529,0 -13530,0 -13531,0 -13532,0 -13533,0 -13534,0 -13535,0 -13536,0 -13537,0 -13538,0 -13539,0 -13540,0 -13541,0 -13542,0 -13543,0 -13544,0 -13545,0 -13546,0 -13547,0 -13548,0 -13549,0 -13550,0 -13551,0 -13552,0 -13553,0 -13554,0 -13555,0 -13556,0 -13557,0 -13558,0 -13559,0 -13560,0 -13561,0 -13562,0 -13563,0 -13564,0 -13565,0 -13566,0 -13567,0 -13568,0 -13569,0 -13570,0 -13571,0 -13572,0 -13573,0 -13574,0 -13575,0 -13576,0 -13577,0 -13578,0 -13579,0 -13580,0 -13581,0 -13582,0 -13583,0 -13584,0 -13585,0 -13586,0 -13587,0 -13588,0 -13589,0 -13590,0 -13591,0 -13592,0 -13593,0 -13594,0 -13595,0 -13596,0 -13597,0 -13598,0 -13599,0 -13600,0 -13601,0 -13602,0 -13603,0 -13604,0 -13605,0 -13606,0 -13607,0 -13608,0 -13609,0 -13610,0 -13611,0 -13612,0 -13613,0 -13614,0 -13615,0 -13616,0 -13617,0 -13618,0 -13619,0 -13620,0 -13621,0 -13622,0 -13623,0 -13624,0 -13625,0 -13626,0 -13627,0 -13628,0 -13629,0 -13630,0 -13631,0 -13632,0 -13633,0 -13634,0 -13635,0 -13636,0 -13637,0 -13638,0 -13639,0 -13640,0 -13641,0 -13642,0 -13643,0 -13644,0 -13645,0 -13646,0 -13647,0 -13648,0 -13649,0 -13650,0 -13651,0 -13652,0 -13653,0 -13654,0 -13655,0 -13656,0 -13657,0 -13658,0 -13659,0 -13660,0 -13661,0 -13662,0 -13663,0 -13664,0 -13665,0 -13666,0 -13667,0 -13668,0 -13669,0 -13670,0 -13671,0 -13672,0 -13673,0 -13674,0 -13675,0 -13676,0 -13677,0 -13678,0 -13679,0 -13680,0 -13681,0 -13682,0 -13683,0 -13684,0 -13685,0 -13686,0 -13687,0 -13688,0 -13689,0 -13690,0 -13691,0 -13692,0 -13693,0 -13694,0 -13695,0 -13696,0 -13697,0 -13698,0 -13699,0 -13700,0 -13701,0 -13702,0 -13703,0 -13704,0 -13705,0 -13706,0 -13707,0 -13708,0 -13709,0 -13710,0 -13711,0 -13712,0 -13713,0 -13714,0 -13715,0 -13716,0 -13717,0 -13718,0 -13719,0 -13720,0 -13721,0 -13722,0 -13723,0 -13724,0 -13725,0 -13726,0 -13727,0 -13728,0 -13729,0 -13730,0 -13731,0 -13732,0 -13733,0 -13734,0 -13735,0 -13736,0 -13737,0 -13738,0 -13739,0 -13740,0 -13741,0 -13742,0 -13743,0 -13744,0 -13745,0 -13746,0 -13747,0 -13748,0 -13749,0 -13750,0 -13751,0 -13752,0 -13753,0 -13754,0 -13755,0 -13756,0 -13757,0 -13758,0 -13759,0 -13760,0 -13761,0 -13762,0 -13763,0 -13764,0 -13765,0 -13766,0 -13767,0 -13768,0 -13769,0 -13770,0 -13771,0 -13772,0 -13773,0 -13774,0 -13775,0 -13776,0 -13777,0 -13778,0 -13779,0 -13780,0 -13781,0 -13782,0 -13783,0 -13784,0 -13785,0 -13786,0 -13787,0 -13788,0 -13789,0 -13790,0 -13791,0 -13792,0 -13793,0 -13794,0 -13795,0 -13796,0 -13797,0 -13798,0 -13799,0 -13800,0 -13801,0 -13802,0 -13803,0 -13804,0 -13805,0 -13806,0 -13807,0 -13808,0 -13809,0 -13810,0 -13811,0 -13812,0 -13813,0 -13814,0 -13815,0 -13816,0 -13817,0 -13818,0 -13819,0 -13820,0 -13821,0 -13822,0 -13823,0 -13824,0 -13825,0 -13826,0 -13827,0 -13828,0 -13829,0 -13830,0 -13831,0 -13832,0 -13833,0 -13834,0 -13835,0 -13836,0 -13837,0 -13838,0 -13839,0 -13840,0 -13841,0 -13842,0 -13843,0 -13844,0 -13845,0 -13846,0 -13847,0 -13848,0 -13849,0 -13850,0 -13851,0 -13852,0 -13853,0 -13854,0 -13855,0 -13856,0 -13857,0 -13858,0 -13859,0 -13860,0 -13861,0 -13862,0 -13863,0 -13864,0 -13865,0 -13866,0 -13867,0 -13868,0 -13869,0 -13870,0 -13871,0 -13872,0 -13873,0 -13874,0 -13875,0 -13876,0 -13877,0 -13878,0 -13879,0 -13880,0 -13881,0 -13882,0 -13883,0 -13884,0 -13885,0 -13886,0 -13887,0 -13888,0 -13889,0 -13890,0 -13891,0 -13892,0 -13893,0 -13894,0 -13895,0 -13896,0 -13897,0 -13898,0 -13899,0 -13900,0 -13901,0 -13902,0 -13903,0 -13904,0 -13905,0 -13906,0 -13907,0 -13908,0 -13909,0 -13910,0 -13911,0 -13912,0 -13913,0 -13914,0 -13915,0 -13916,0 -13917,0 -13918,0 -13919,0 -13920,0 -13921,0 -13922,0 -13923,0 -13924,0 -13925,0 -13926,0 -13927,0 -13928,0 -13929,0 -13930,0 -13931,0 -13932,0 -13933,0 -13934,0 -13935,0 -13936,0 -13937,0 -13938,0 -13939,0 -13940,0 -13941,0 -13942,0 -13943,0 -13944,0 -13945,0 -13946,0 -13947,0 -13948,0 -13949,0 -13950,0 -13951,0 -13952,0 -13953,0 -13954,0 -13955,0 -13956,0 -13957,0 -13958,0 -13959,0 -13960,0 -13961,0 -13962,0 -13963,0 -13964,0 -13965,0 -13966,0 -13967,0 -13968,0 -13969,0 -13970,0 -13971,0 -13972,0 -13973,0 -13974,0 -13975,0 -13976,0 -13977,0 -13978,0 -13979,0 -13980,0 -13981,0 -13982,0 -13983,0 -13984,0 -13985,0 -13986,0 -13987,0 -13988,0 -13989,0 -13990,0 -13991,0 -13992,0 -13993,0 -13994,0 -13995,0 -13996,0 -13997,0 -13998,0 -13999,0 -14000,0 -14001,0 -14002,0 -14003,0 -14004,0 -14005,0 -14006,0 -14007,0 -14008,0 -14009,0 -14010,0 -14011,0 -14012,0 -14013,0 -14014,0 -14015,0 -14016,0 -14017,0 -14018,0 -14019,0 -14020,0 -14021,0 -14022,0 -14023,0 -14024,0 -14025,0 -14026,0 -14027,0 -14028,0 -14029,0 -14030,0 -14031,0 -14032,0 -14033,0 -14034,0 -14035,0 -14036,0 -14037,0 -14038,0 -14039,0 -14040,0 -14041,0 -14042,0 -14043,0 -14044,0 -14045,0 -14046,0 -14047,0 -14048,0 -14049,0 -14050,0 -14051,0 -14052,0 -14053,0 -14054,0 -14055,0 -14056,0 -14057,0 -14058,0 -14059,0 -14060,0 -14061,0 -14062,0 -14063,0 -14064,0 -14065,0 -14066,0 -14067,0 -14068,0 -14069,0 -14070,0 -14071,0 -14072,0 -14073,0 -14074,0 -14075,0 -14076,0 -14077,0 -14078,0 -14079,0 -14080,0 -14081,0 -14082,0 -14083,0 -14084,0 -14085,0 -14086,0 -14087,0 -14088,0 -14089,0 -14090,0 -14091,0 -14092,0 -14093,0 -14094,0 -14095,0 -14096,0 -14097,0 -14098,0 -14099,0 -14100,0 -14101,0 -14102,0 -14103,0 -14104,0 -14105,0 -14106,0 -14107,0 -14108,0 -14109,0 -14110,0 -14111,0 -14112,0 -14113,0 -14114,0 -14115,0 -14116,0 -14117,0 -14118,0 -14119,0 -14120,0 -14121,0 -14122,0 -14123,0 -14124,0 -14125,0 -14126,0 -14127,0 -14128,0 -14129,0 -14130,0 -14131,0 -14132,0 -14133,0 -14134,0 -14135,0 -14136,0 -14137,0 -14138,0 -14139,0 -14140,0 -14141,0 -14142,0 -14143,0 -14144,0 -14145,0 -14146,0 -14147,0 -14148,0 -14149,0 -14150,0 -14151,0 -14152,0 -14153,0 -14154,0 -14155,0 -14156,0 -14157,0 -14158,0 -14159,0 -14160,0 -14161,0 -14162,0 -14163,0 -14164,0 -14165,0 -14166,0 -14167,0 -14168,0 -14169,0 -14170,0 -14171,0 -14172,0 -14173,0 -14174,0 -14175,0 -14176,0 -14177,0 -14178,0 -14179,0 -14180,0 -14181,0 -14182,0 -14183,0 -14184,0 -14185,0 -14186,0 -14187,0 -14188,0 -14189,0 -14190,0 -14191,0 -14192,0 -14193,0 -14194,0 -14195,0 -14196,0 -14197,0 -14198,0 -14199,0 -14200,0 -14201,0 -14202,0 -14203,0 -14204,0 -14205,0 -14206,0 -14207,0 -14208,0 -14209,0 -14210,0 -14211,0 -14212,0 -14213,0 -14214,0 -14215,0 -14216,0 -14217,0 -14218,0 -14219,0 -14220,0 -14221,0 -14222,0 -14223,0 -14224,0 -14225,0 -14226,0 -14227,0 -14228,0 -14229,0 -14230,0 -14231,0 -14232,0 -14233,0 -14234,0 -14235,0 -14236,0 -14237,0 -14238,0 -14239,0 -14240,0 -14241,0 -14242,0 -14243,0 -14244,0 -14245,0 -14246,0 -14247,0 -14248,0 -14249,0 -14250,0 -14251,0 -14252,0 -14253,0 -14254,0 -14255,0 -14256,0 -14257,0 -14258,0 -14259,0 -14260,0 -14261,0 -14262,0 -14263,0 -14264,0 -14265,0 -14266,0 -14267,0 -14268,0 -14269,0 -14270,0 -14271,0 -14272,0 -14273,0 -14274,0 -14275,0 -14276,0 -14277,0 -14278,0 -14279,0 -14280,0 -14281,0 -14282,0 -14283,0 -14284,0 -14285,0 -14286,0 -14287,0 -14288,0 -14289,0 -14290,0 -14291,0 -14292,0 -14293,0 -14294,0 -14295,0 -14296,0 -14297,0 -14298,0 -14299,0 -14300,0 -14301,0 -14302,0 -14303,0 -14304,0 -14305,0 -14306,0 -14307,0 -14308,0 -14309,0 -14310,0 -14311,0 -14312,0 -14313,0 -14314,0 -14315,0 -14316,0 -14317,0 -14318,0 -14319,0 -14320,0 -14321,0 -14322,0 -14323,0 -14324,0 -14325,0 -14326,0 -14327,0 -14328,0 -14329,0 -14330,0 -14331,0 -14332,0 -14333,0 -14334,0 -14335,0 -14336,0 -14337,0 -14338,0 -14339,0 -14340,0 -14341,0 -14342,0 -14343,0 -14344,0 -14345,0 -14346,0 -14347,0 -14348,0 -14349,0 -14350,0 -14351,0 -14352,0 -14353,0 -14354,0 -14355,0 -14356,0 -14357,0 -14358,0 -14359,0 -14360,0 -14361,0 -14362,0 -14363,0 -14364,0 -14365,0 -14366,0 -14367,0 -14368,0 -14369,0 -14370,0 -14371,0 -14372,0 -14373,0 -14374,0 -14375,0 -14376,0 -14377,0 -14378,0 -14379,0 -14380,0 -14381,0 -14382,0 -14383,0 -14384,0 -14385,0 -14386,0 -14387,0 -14388,0 -14389,0 -14390,0 -14391,0 -14392,0 -14393,0 -14394,0 -14395,0 -14396,0 -14397,0 -14398,0 -14399,0 -14400,0 -14401,0 -14402,0 -14403,0 -14404,0 -14405,0 -14406,0 -14407,0 -14408,0 -14409,0 -14410,0 -14411,0 -14412,0 -14413,0 -14414,0 -14415,0 -14416,0 -14417,0 -14418,0 -14419,0 -14420,0 -14421,0 -14422,0 -14423,0 -14424,0 -14425,0 -14426,0 -14427,0 -14428,0 -14429,0 -14430,0 -14431,0 -14432,0 -14433,0 -14434,0 -14435,0 -14436,0 -14437,0 -14438,0 -14439,0 -14440,0 -14441,0 -14442,0 -14443,0 -14444,0 -14445,0 -14446,0 -14447,0 -14448,0 -14449,0 -14450,0 -14451,0 -14452,0 -14453,0 -14454,0 -14455,0 -14456,0 -14457,0 -14458,0 -14459,0 -14460,0 -14461,0 -14462,0 -14463,0 -14464,0 -14465,0 -14466,0 -14467,0 -14468,0 -14469,0 -14470,0 -14471,0 -14472,0 -14473,0 -14474,0 -14475,0 -14476,0 -14477,0 -14478,0 -14479,0 -14480,0 -14481,0 -14482,0 -14483,0 -14484,0 -14485,0 -14486,0 -14487,0 -14488,0 -14489,0 -14490,0 -14491,0 -14492,0 -14493,0 -14494,0 -14495,0 -14496,0 -14497,0 -14498,0 -14499,0 -14500,0 -14501,0 -14502,0 -14503,0 -14504,0 -14505,0 -14506,0 -14507,0 -14508,0 -14509,0 -14510,0 -14511,0 -14512,0 -14513,0 -14514,0 -14515,0 -14516,0 -14517,0 -14518,0 -14519,0 -14520,0 -14521,0 -14522,0 -14523,0 -14524,0 -14525,0 -14526,0 -14527,0 -14528,0 -14529,0 -14530,0 -14531,0 -14532,0 -14533,0 -14534,0 -14535,0 -14536,0 -14537,0 -14538,0 -14539,0 -14540,0 -14541,0 -14542,0 -14543,0 -14544,0 -14545,0 -14546,0 -14547,0 -14548,0 -14549,0 -14550,0 -14551,0 -14552,0 -14553,0 -14554,0 -14555,0 -14556,0 -14557,0 -14558,0 -14559,0 -14560,0 -14561,0 -14562,0 -14563,0 -14564,0 -14565,0 -14566,0 -14567,0 -14568,0 -14569,0 -14570,0 -14571,0 -14572,0 -14573,0 -14574,0 -14575,0 -14576,0 -14577,0 -14578,0 -14579,0 -14580,0 -14581,0 -14582,0 -14583,0 -14584,0 -14585,0 -14586,0 -14587,0 -14588,0 -14589,0 -14590,0 -14591,0 -14592,0 -14593,0 -14594,0 -14595,0 -14596,0 -14597,0 -14598,0 -14599,0 -14600,0 -14601,0 -14602,0 -14603,0 -14604,0 -14605,0 -14606,0 -14607,0 -14608,0 -14609,0 -14610,0 -14611,0 -14612,0 -14613,0 -14614,0 -14615,0 -14616,0 -14617,0 -14618,0 -14619,0 -14620,0 -14621,0 -14622,0 -14623,0 -14624,0 -14625,0 -14626,0 -14627,0 -14628,0 -14629,0 -14630,0 -14631,0 -14632,0 -14633,0 -14634,0 -14635,0 -14636,0 -14637,0 -14638,0 -14639,0 -14640,0 -14641,0 -14642,0 -14643,0 -14644,0 -14645,0 -14646,0 -14647,0 -14648,0 -14649,0 -14650,0 -14651,0 -14652,0 -14653,0 -14654,0 -14655,0 -14656,0 -14657,0 -14658,0 -14659,0 -14660,0 -14661,0 -14662,0 -14663,0 -14664,0 -14665,0 -14666,0 -14667,0 -14668,0 -14669,0 -14670,0 -14671,0 -14672,0 -14673,0 -14674,0 -14675,0 -14676,0 -14677,0 -14678,0 -14679,0 -14680,0 -14681,0 -14682,0 -14683,0 -14684,0 -14685,0 -14686,0 -14687,0 -14688,0 -14689,0 -14690,0 -14691,0 -14692,0 -14693,0 -14694,0 -14695,0 -14696,0 -14697,0 -14698,0 -14699,0 -14700,0 -14701,0 -14702,0 -14703,0 -14704,0 -14705,0 -14706,0 -14707,0 -14708,0 -14709,0 -14710,0 -14711,0 -14712,0 -14713,0 -14714,0 -14715,0 -14716,0 -14717,0 -14718,0 -14719,0 -14720,0 -14721,0 -14722,0 -14723,0 -14724,0 -14725,0 -14726,0 -14727,0 -14728,0 -14729,0 -14730,0 -14731,0 -14732,0 -14733,0 -14734,0 -14735,0 -14736,0 -14737,0 -14738,0 -14739,0 -14740,0 -14741,0 -14742,0 -14743,0 -14744,0 -14745,0 -14746,0 -14747,0 -14748,0 -14749,0 -14750,0 -14751,0 -14752,0 -14753,0 -14754,0 -14755,0 -14756,0 -14757,0 -14758,0 -14759,0 -14760,0 -14761,0 -14762,0 -14763,0 -14764,0 -14765,0 -14766,0 -14767,0 -14768,0 -14769,0 -14770,0 -14771,0 -14772,0 -14773,0 -14774,0 -14775,0 -14776,0 -14777,0 -14778,0 -14779,0 -14780,0 -14781,0 -14782,0 -14783,0 -14784,0 -14785,0 -14786,0 -14787,0 -14788,0 -14789,0 -14790,0 -14791,0 -14792,0 -14793,0 -14794,0 -14795,0 -14796,0 -14797,0 -14798,0 -14799,0 -14800,0 -14801,0 -14802,0 -14803,0 -14804,0 -14805,0 -14806,0 -14807,0 -14808,0 -14809,0 -14810,0 -14811,0 -14812,0 -14813,0 -14814,0 -14815,0 -14816,0 -14817,0 -14818,0 -14819,0 -14820,0 -14821,0 -14822,0 -14823,0 -14824,0 -14825,0 -14826,0 -14827,0 -14828,0 -14829,0 -14830,0 -14831,0 -14832,0 -14833,0 -14834,0 -14835,0 -14836,0 -14837,0 -14838,0 -14839,0 -14840,0 -14841,0 -14842,0 -14843,0 -14844,0 -14845,0 -14846,0 -14847,0 -14848,0 -14849,0 -14850,0 -14851,0 -14852,0 -14853,0 -14854,0 -14855,0 -14856,0 -14857,0 -14858,0 -14859,0 -14860,0 -14861,0 -14862,0 -14863,0 -14864,0 -14865,0 -14866,0 -14867,0 -14868,0 -14869,0 -14870,0 -14871,0 -14872,0 -14873,0 -14874,0 -14875,0 -14876,0 -14877,0 -14878,0 -14879,0 -14880,0 -14881,0 -14882,0 -14883,0 -14884,0 -14885,0 -14886,0 -14887,0 -14888,0 -14889,0 -14890,0 -14891,0 -14892,0 -14893,0 -14894,0 -14895,0 -14896,0 -14897,0 -14898,0 -14899,0 -14900,0 -14901,0 -14902,0 -14903,0 -14904,0 -14905,0 -14906,0 -14907,0 -14908,0 -14909,0 -14910,0 -14911,0 -14912,0 -14913,0 -14914,0 -14915,0 -14916,0 -14917,0 -14918,0 -14919,0 -14920,0 -14921,0 -14922,0 -14923,0 -14924,0 -14925,0 -14926,0 -14927,0 -14928,0 -14929,0 -14930,0 -14931,0 -14932,0 -14933,0 -14934,0 -14935,0 -14936,0 -14937,0 -14938,0 -14939,0 -14940,0 -14941,0 -14942,0 -14943,0 -14944,0 -14945,0 -14946,0 -14947,0 -14948,0 -14949,0 -14950,0 -14951,0 -14952,0 -14953,0 -14954,0 -14955,0 -14956,0 -14957,0 -14958,0 -14959,0 -14960,0 -14961,0 -14962,0 -14963,0 -14964,0 -14965,0 -14966,0 -14967,0 -14968,0 -14969,0 -14970,0 -14971,0 -14972,0 -14973,0 -14974,0 -14975,0 -14976,0 -14977,0 -14978,0 -14979,0 -14980,0 -14981,0 -14982,0 -14983,0 -14984,0 -14985,0 -14986,0 -14987,0 -14988,0 -14989,0 -14990,0 -14991,0 -14992,0 -14993,0 -14994,0 -14995,0 -14996,0 -14997,0 -14998,0 -14999,0 -15000,0 -15001,0 -15002,0 -15003,0 -15004,0 -15005,0 -15006,0 -15007,0 -15008,0 -15009,0 -15010,0 -15011,0 -15012,0 -15013,0 -15014,0 -15015,0 -15016,0 -15017,0 -15018,0 -15019,0 -15020,0 -15021,0 -15022,0 -15023,0 -15024,0 -15025,0 -15026,0 -15027,0 -15028,0 -15029,0 -15030,0 -15031,0 -15032,0 -15033,0 -15034,0 -15035,0 -15036,0 -15037,0 -15038,0 -15039,0 -15040,0 -15041,0 -15042,0 -15043,0 -15044,0 -15045,0 -15046,0 -15047,0 -15048,0 -15049,0 -15050,0 -15051,0 -15052,0 -15053,0 -15054,0 -15055,0 -15056,0 -15057,0 -15058,0 -15059,0 -15060,0 -15061,0 -15062,0 -15063,0 -15064,0 -15065,0 -15066,0 -15067,0 -15068,0 -15069,0 -15070,0 -15071,0 -15072,0 -15073,0 -15074,0 -15075,0 -15076,0 -15077,0 -15078,0 -15079,0 -15080,0 -15081,0 -15082,0 -15083,0 -15084,0 -15085,0 -15086,0 -15087,0 -15088,0 -15089,0 -15090,0 -15091,0 -15092,0 -15093,0 -15094,0 -15095,0 -15096,0 -15097,0 -15098,0 -15099,0 -15100,0 -15101,0 -15102,0 -15103,0 -15104,0 -15105,0 -15106,0 -15107,0 -15108,0 -15109,0 -15110,0 -15111,0 -15112,0 -15113,0 -15114,0 -15115,0 -15116,0 -15117,0 -15118,0 -15119,0 -15120,0 -15121,0 -15122,0 -15123,0 -15124,0 -15125,0 -15126,0 -15127,0 -15128,0 -15129,0 -15130,0 -15131,0 -15132,0 -15133,0 -15134,0 -15135,0 -15136,0 -15137,0 -15138,0 -15139,0 -15140,0 -15141,0 -15142,0 -15143,0 -15144,0 -15145,0 -15146,0 -15147,0 -15148,0 -15149,0 -15150,0 -15151,0 -15152,0 -15153,0 -15154,0 -15155,0 -15156,0 -15157,0 -15158,0 -15159,0 -15160,0 -15161,0 -15162,0 -15163,0 -15164,0 -15165,0 -15166,0 -15167,0 -15168,0 -15169,0 -15170,0 -15171,0 -15172,0 -15173,0 -15174,0 -15175,0 -15176,0 -15177,0 -15178,0 -15179,0 -15180,0 -15181,0 -15182,0 -15183,0 -15184,0 -15185,0 -15186,0 -15187,0 -15188,0 -15189,0 -15190,0 -15191,0 -15192,0 -15193,0 -15194,0 -15195,0 -15196,0 -15197,0 -15198,0 -15199,0 -15200,0 -15201,0 -15202,0 -15203,0 -15204,0 -15205,0 -15206,0 -15207,0 -15208,0 -15209,0 -15210,0 -15211,0 -15212,0 -15213,0 -15214,0 -15215,0 -15216,0 -15217,0 -15218,0 -15219,0 -15220,0 -15221,0 -15222,0 -15223,0 -15224,0 -15225,0 -15226,0 -15227,0 -15228,0 -15229,0 -15230,0 -15231,0 -15232,0 -15233,0 -15234,0 -15235,0 -15236,0 -15237,0 -15238,0 -15239,0 -15240,0 -15241,0 -15242,0 -15243,0 -15244,0 -15245,0 -15246,0 -15247,0 -15248,0 -15249,0 -15250,0 -15251,0 -15252,0 -15253,0 -15254,0 -15255,0 -15256,0 -15257,0 -15258,0 -15259,0 -15260,0 -15261,0 -15262,0 -15263,0 -15264,0 -15265,0 -15266,0 -15267,0 -15268,0 -15269,0 -15270,0 -15271,0 -15272,0 -15273,0 -15274,0 -15275,0 -15276,0 -15277,0 -15278,0 -15279,0 -15280,0 -15281,0 -15282,0 -15283,0 -15284,0 -15285,0 -15286,0 -15287,0 -15288,0 -15289,0 -15290,0 -15291,0 -15292,0 -15293,0 -15294,0 -15295,0 -15296,0 -15297,0 -15298,0 -15299,0 -15300,0 -15301,0 -15302,0 -15303,0 -15304,0 -15305,0 -15306,0 -15307,0 -15308,0 -15309,0 -15310,0 -15311,0 -15312,0 -15313,0 -15314,0 -15315,0 -15316,0 -15317,0 -15318,0 -15319,0 -15320,0 -15321,0 -15322,0 -15323,0 -15324,0 -15325,0 -15326,0 -15327,0 -15328,0 -15329,0 -15330,0 -15331,0 -15332,0 -15333,0 -15334,0 -15335,0 -15336,0 -15337,0 -15338,0 -15339,0 -15340,0 -15341,0 -15342,0 -15343,0 -15344,0 -15345,0 -15346,0 -15347,0 -15348,0 -15349,0 -15350,0 -15351,0 -15352,0 -15353,0 -15354,0 -15355,0 -15356,0 -15357,0 -15358,0 -15359,0 -15360,0 -15361,0 -15362,0 -15363,0 -15364,0 -15365,0 -15366,0 -15367,0 -15368,0 -15369,0 -15370,0 -15371,0 -15372,0 -15373,0 -15374,0 -15375,0 -15376,0 -15377,0 -15378,0 -15379,0 -15380,0 -15381,0 -15382,0 -15383,0 -15384,0 -15385,0 -15386,0 -15387,0 -15388,0 -15389,0 -15390,0 -15391,0 -15392,0 -15393,0 -15394,0 -15395,0 -15396,0 -15397,0 -15398,0 -15399,0 -15400,0 -15401,0 -15402,0 -15403,0 -15404,0 -15405,0 -15406,0 -15407,0 -15408,0 -15409,0 -15410,0 -15411,0 -15412,0 -15413,0 -15414,0 -15415,0 -15416,0 -15417,0 -15418,0 -15419,0 -15420,0 -15421,0 -15422,0 -15423,0 -15424,0 -15425,0 -15426,0 -15427,0 -15428,0 -15429,0 -15430,0 -15431,0 -15432,0 -15433,0 -15434,0 -15435,0 -15436,0 -15437,0 -15438,0 -15439,0 -15440,0 -15441,0 -15442,0 -15443,0 -15444,0 -15445,0 -15446,0 -15447,0 -15448,0 -15449,0 -15450,0 -15451,0 -15452,0 -15453,0 -15454,0 -15455,0 -15456,0 -15457,0 -15458,0 -15459,0 -15460,0 -15461,0 -15462,0 -15463,0 -15464,0 -15465,0 -15466,0 -15467,0 -15468,0 -15469,0 -15470,0 -15471,0 -15472,0 -15473,0 -15474,0 -15475,0 -15476,0 -15477,0 -15478,0 -15479,0 -15480,0 -15481,0 -15482,0 -15483,0 -15484,0 -15485,0 -15486,0 -15487,0 -15488,0 -15489,0 -15490,0 -15491,0 -15492,0 -15493,0 -15494,0 -15495,0 -15496,0 -15497,0 -15498,0 -15499,0 -15500,0 -15501,0 -15502,0 -15503,0 -15504,0 -15505,0 -15506,0 -15507,0 -15508,0 -15509,0 -15510,0 -15511,0 -15512,0 -15513,0 -15514,0 -15515,0 -15516,0 -15517,0 -15518,0 -15519,0 -15520,0 -15521,0 -15522,0 -15523,0 -15524,0 -15525,0 -15526,0 -15527,0 -15528,0 -15529,0 -15530,0 -15531,0 -15532,0 -15533,0 -15534,0 -15535,0 -15536,0 -15537,0 -15538,0 -15539,0 -15540,0 -15541,0 -15542,0 -15543,0 -15544,0 -15545,0 -15546,0 -15547,0 -15548,0 -15549,0 -15550,0 -15551,0 -15552,0 -15553,0 -15554,0 -15555,0 -15556,0 -15557,0 -15558,0 -15559,0 -15560,0 -15561,0 -15562,0 -15563,0 -15564,0 -15565,0 -15566,0 -15567,0 -15568,0 -15569,0 -15570,0 -15571,0 -15572,0 -15573,0 -15574,0 -15575,0 -15576,0 -15577,0 -15578,0 -15579,0 -15580,0 -15581,0 -15582,0 -15583,0 -15584,0 -15585,0 -15586,0 -15587,0 -15588,0 -15589,0 -15590,0 -15591,0 -15592,0 -15593,0 -15594,0 -15595,0 -15596,0 -15597,0 -15598,0 -15599,0 -15600,0 -15601,0 -15602,0 -15603,0 -15604,0 -15605,0 -15606,0 -15607,0 -15608,0 -15609,0 -15610,0 -15611,0 -15612,0 -15613,0 -15614,0 -15615,0 -15616,0 -15617,0 -15618,0 -15619,0 -15620,0 -15621,0 -15622,0 -15623,0 -15624,0 -15625,0 -15626,0 -15627,0 -15628,0 -15629,0 -15630,0 -15631,0 -15632,0 -15633,0 -15634,0 -15635,0 -15636,0 -15637,0 -15638,0 -15639,0 -15640,0 -15641,0 -15642,0 -15643,0 -15644,0 -15645,0 -15646,0 -15647,0 -15648,0 -15649,0 -15650,0 -15651,0 -15652,0 -15653,0 -15654,0 -15655,0 -15656,0 -15657,0 -15658,0 -15659,0 -15660,0 -15661,0 -15662,0 -15663,0 -15664,0 -15665,0 -15666,0 -15667,0 -15668,0 -15669,0 -15670,0 -15671,0 -15672,0 -15673,0 -15674,0 -15675,0 -15676,0 -15677,0 -15678,0 -15679,0 -15680,0 -15681,0 -15682,0 -15683,0 -15684,0 -15685,0 -15686,0 -15687,0 -15688,0 -15689,0 -15690,0 -15691,0 -15692,0 -15693,0 -15694,0 -15695,0 -15696,0 -15697,0 -15698,0 -15699,0 -15700,0 -15701,0 -15702,0 -15703,0 -15704,0 -15705,0 -15706,0 -15707,0 -15708,0 -15709,0 -15710,0 -15711,0 -15712,0 -15713,0 -15714,0 -15715,0 -15716,0 -15717,0 -15718,0 -15719,0 -15720,0 -15721,0 -15722,0 -15723,0 -15724,0 -15725,0 -15726,0 -15727,0 -15728,0 -15729,0 -15730,0 -15731,0 -15732,0 -15733,0 -15734,0 -15735,0 -15736,0 -15737,0 -15738,0 -15739,0 -15740,0 -15741,0 -15742,0 -15743,0 -15744,0 -15745,0 -15746,0 -15747,0 -15748,0 -15749,0 -15750,0 -15751,0 -15752,0 -15753,0 -15754,0 -15755,0 -15756,0 -15757,0 -15758,0 -15759,0 -15760,0 -15761,0 -15762,0 -15763,0 -15764,0 -15765,0 -15766,0 -15767,0 -15768,0 -15769,0 -15770,0 -15771,0 -15772,0 -15773,0 -15774,0 -15775,0 -15776,0 -15777,0 -15778,0 -15779,0 -15780,0 -15781,0 -15782,0 -15783,0 -15784,0 -15785,0 -15786,0 -15787,0 -15788,0 -15789,0 -15790,0 -15791,0 -15792,0 -15793,0 -15794,0 -15795,0 -15796,0 -15797,0 -15798,0 -15799,0 -15800,0 -15801,0 -15802,0 -15803,0 -15804,0 -15805,0 -15806,0 -15807,0 -15808,0 -15809,0 -15810,0 -15811,0 -15812,0 -15813,0 -15814,0 -15815,0 -15816,0 -15817,0 -15818,0 -15819,0 -15820,0 -15821,0 -15822,0 -15823,0 -15824,0 -15825,0 -15826,0 -15827,0 -15828,0 -15829,0 -15830,0 -15831,0 -15832,0 -15833,0 -15834,0 -15835,0 -15836,0 -15837,0 -15838,0 -15839,0 -15840,0 -15841,0 -15842,0 -15843,0 -15844,0 -15845,0 -15846,0 -15847,0 -15848,0 -15849,0 -15850,0 -15851,0 -15852,0 -15853,0 -15854,0 -15855,0 -15856,0 -15857,0 -15858,0 -15859,0 -15860,0 -15861,0 -15862,0 -15863,0 -15864,0 -15865,0 -15866,0 -15867,0 -15868,0 -15869,0 -15870,0 -15871,0 -15872,0 -15873,0 -15874,0 -15875,0 -15876,0 -15877,0 -15878,0 -15879,0 -15880,0 -15881,0 -15882,0 -15883,0 -15884,0 -15885,0 -15886,0 -15887,0 -15888,0 -15889,0 -15890,0 -15891,0 -15892,0 -15893,0 -15894,0 -15895,0 -15896,0 -15897,0 -15898,0 -15899,0 -15900,0 -15901,0 -15902,0 -15903,0 -15904,0 -15905,0 -15906,0 -15907,0 -15908,0 -15909,0 -15910,0 -15911,0 -15912,0 -15913,0 -15914,0 -15915,0 -15916,0 -15917,0 -15918,0 -15919,0 -15920,0 -15921,0 -15922,0 -15923,0 -15924,0 -15925,0 -15926,0 -15927,0 -15928,0 -15929,0 -15930,0 -15931,0 -15932,0 -15933,0 -15934,0 -15935,0 -15936,0 -15937,0 -15938,0 -15939,0 -15940,0 -15941,0 -15942,0 -15943,0 -15944,0 -15945,0 -15946,0 -15947,0 -15948,0 -15949,0 -15950,0 -15951,0 -15952,0 -15953,0 -15954,0 -15955,0 -15956,0 -15957,0 -15958,0 -15959,0 -15960,0 -15961,0 -15962,0 -15963,0 -15964,0 -15965,0 -15966,0 -15967,0 -15968,0 -15969,0 -15970,0 -15971,0 -15972,0 -15973,0 -15974,0 -15975,0 -15976,0 -15977,0 -15978,0 -15979,0 -15980,0 -15981,0 -15982,0 -15983,0 -15984,0 -15985,0 -15986,0 -15987,0 -15988,0 -15989,0 -15990,0 -15991,0 -15992,0 -15993,0 -15994,0 -15995,0 -15996,0 -15997,0 -15998,0 -15999,0 -16000,0 -16001,0 -16002,0 -16003,0 -16004,0 -16005,0 -16006,0 -16007,0 -16008,0 -16009,0 -16010,0 -16011,0 -16012,0 -16013,0 -16014,0 -16015,0 -16016,0 -16017,0 -16018,0 -16019,0 -16020,0 -16021,0 -16022,0 -16023,0 -16024,0 -16025,0 -16026,0 -16027,0 -16028,0 -16029,0 -16030,0 -16031,0 -16032,0 -16033,0 -16034,0 -16035,0 -16036,0 -16037,0 -16038,0 -16039,0 -16040,0 -16041,0 -16042,0 -16043,0 -16044,0 -16045,0 -16046,0 -16047,0 -16048,0 -16049,0 -16050,0 -16051,0 -16052,0 -16053,0 -16054,0 -16055,0 -16056,0 -16057,0 -16058,0 -16059,0 -16060,0 -16061,0 -16062,0 -16063,0 -16064,0 -16065,0 -16066,0 -16067,0 -16068,0 -16069,0 -16070,0 -16071,0 -16072,0 -16073,0 -16074,0 -16075,0 -16076,0 -16077,0 -16078,0 -16079,0 -16080,0 -16081,0 -16082,0 -16083,0 -16084,0 -16085,0 -16086,0 -16087,0 -16088,0 -16089,0 -16090,0 -16091,0 -16092,0 -16093,0 -16094,0 -16095,0 -16096,0 -16097,0 -16098,0 -16099,0 -16100,0 -16101,0 -16102,0 -16103,0 -16104,0 -16105,0 -16106,0 -16107,0 -16108,0 -16109,0 -16110,0 -16111,0 -16112,0 -16113,0 -16114,0 -16115,0 -16116,0 -16117,0 -16118,0 -16119,0 -16120,0 -16121,0 -16122,0 -16123,0 -16124,0 -16125,0 -16126,0 -16127,0 -16128,0 -16129,0 -16130,0 -16131,0 -16132,0 -16133,0 -16134,0 -16135,0 -16136,0 -16137,0 -16138,0 -16139,0 -16140,0 -16141,0 -16142,0 -16143,0 -16144,0 -16145,0 -16146,0 -16147,0 -16148,0 -16149,0 -16150,0 -16151,0 -16152,0 -16153,0 -16154,0 -16155,0 -16156,0 -16157,0 -16158,0 -16159,0 -16160,0 -16161,0 -16162,0 -16163,0 -16164,0 -16165,0 -16166,0 -16167,0 -16168,0 -16169,0 -16170,0 -16171,0 -16172,0 -16173,0 -16174,0 -16175,0 -16176,0 -16177,0 -16178,0 -16179,0 -16180,0 -16181,0 -16182,0 -16183,0 -16184,0 -16185,0 -16186,0 -16187,0 -16188,0 -16189,0 -16190,0 -16191,0 -16192,0 -16193,0 -16194,0 -16195,0 -16196,0 -16197,0 -16198,0 -16199,0 -16200,0 -16201,0 -16202,0 -16203,0 -16204,0 -16205,0 -16206,0 -16207,0 -16208,0 -16209,0 -16210,0 -16211,0 -16212,0 -16213,0 -16214,0 -16215,0 -16216,0 -16217,0 -16218,0 -16219,0 -16220,0 -16221,0 -16222,0 -16223,0 -16224,0 -16225,0 -16226,0 -16227,0 -16228,0 -16229,0 -16230,0 -16231,0 -16232,0 -16233,0 -16234,0 -16235,0 -16236,0 -16237,0 -16238,0 -16239,0 -16240,0 -16241,0 -16242,0 -16243,0 -16244,0 -16245,0 -16246,0 -16247,0 -16248,0 -16249,0 -16250,0 -16251,0 -16252,0 -16253,0 -16254,0 -16255,0 -16256,0 -16257,0 -16258,0 -16259,0 -16260,0 -16261,0 -16262,0 -16263,0 -16264,0 -16265,0 -16266,0 -16267,0 -16268,0 -16269,0 -16270,0 -16271,0 -16272,0 -16273,0 -16274,0 -16275,0 -16276,0 -16277,0 -16278,0 -16279,0 -16280,0 -16281,0 -16282,0 -16283,0 -16284,0 -16285,0 -16286,0 -16287,0 -16288,0 -16289,0 -16290,0 -16291,0 -16292,0 -16293,0 -16294,0 -16295,0 -16296,0 -16297,0 -16298,0 -16299,0 -16300,0 -16301,0 -16302,0 -16303,0 -16304,0 -16305,0 -16306,0 -16307,0 -16308,0 -16309,0 -16310,0 -16311,0 -16312,0 -16313,0 -16314,0 -16315,0 -16316,0 -16317,0 -16318,0 -16319,0 -16320,0 -16321,0 -16322,0 -16323,0 -16324,0 -16325,0 -16326,0 -16327,0 -16328,0 -16329,0 -16330,0 -16331,0 -16332,0 -16333,0 -16334,0 -16335,0 -16336,0 -16337,0 -16338,0 -16339,0 -16340,0 -16341,0 -16342,0 -16343,0 -16344,0 -16345,0 -16346,0 -16347,0 -16348,0 -16349,0 -16350,0 -16351,0 -16352,0 -16353,0 -16354,0 -16355,0 -16356,0 -16357,0 -16358,0 -16359,0 -16360,0 -16361,0 -16362,0 -16363,0 -16364,0 -16365,0 -16366,0 -16367,0 -16368,0 -16369,0 -16370,0 -16371,0 -16372,0 -16373,0 -16374,0 -16375,0 -16376,0 -16377,0 -16378,0 -16379,0 -16380,0 -16381,0 -16382,0 -16383,0 -16384,0 -16385,0 -16386,0 -16387,0 -16388,0 -16389,0 -16390,0 -16391,0 -16392,0 -16393,0 -16394,0 -16395,0 -16396,0 -16397,0 -16398,0 -16399,0 -16400,0 -16401,0 -16402,0 -16403,0 -16404,0 -16405,0 -16406,0 -16407,0 -16408,0 -16409,0 -16410,0 -16411,0 -16412,0 -16413,0 -16414,0 -16415,0 -16416,0 -16417,0 -16418,0 -16419,0 -16420,0 -16421,0 -16422,0 -16423,0 -16424,0 -16425,0 -16426,0 -16427,0 -16428,0 -16429,0 -16430,0 -16431,0 -16432,0 -16433,0 -16434,0 -16435,0 -16436,0 -16437,0 -16438,0 -16439,0 -16440,0 -16441,0 -16442,0 -16443,0 -16444,0 -16445,0 -16446,0 -16447,0 -16448,0 -16449,0 -16450,0 -16451,0 -16452,0 -16453,0 -16454,0 -16455,0 -16456,0 -16457,0 -16458,0 -16459,0 -16460,0 -16461,0 -16462,0 -16463,0 -16464,0 -16465,0 -16466,0 -16467,0 -16468,0 -16469,0 -16470,0 -16471,0 -16472,0 -16473,0 -16474,0 -16475,0 -16476,0 -16477,0 -16478,0 -16479,0 -16480,0 -16481,0 -16482,0 -16483,0 -16484,0 -16485,0 -16486,0 -16487,0 -16488,0 -16489,0 -16490,0 -16491,0 -16492,0 -16493,0 -16494,0 -16495,0 -16496,0 -16497,0 -16498,0 -16499,0 -16500,0 -16501,0 -16502,0 -16503,0 -16504,0 -16505,0 -16506,0 -16507,0 -16508,0 -16509,0 -16510,0 -16511,0 -16512,0 -16513,0 -16514,0 -16515,0 -16516,0 -16517,0 -16518,0 -16519,0 -16520,0 -16521,0 -16522,0 -16523,0 -16524,0 -16525,0 -16526,0 -16527,0 -16528,0 -16529,0 -16530,0 -16531,0 -16532,0 -16533,0 -16534,0 -16535,0 -16536,0 -16537,0 -16538,0 -16539,0 -16540,0 -16541,0 -16542,0 -16543,0 -16544,0 -16545,0 -16546,0 -16547,0 -16548,0 -16549,0 -16550,0 -16551,0 -16552,0 -16553,0 -16554,0 -16555,0 -16556,0 -16557,0 -16558,0 -16559,0 -16560,0 -16561,0 -16562,0 -16563,0 -16564,0 -16565,0 -16566,0 -16567,0 -16568,0 -16569,0 -16570,0 -16571,0 -16572,0 -16573,0 -16574,0 -16575,0 -16576,0 -16577,0 -16578,0 -16579,0 -16580,0 -16581,0 -16582,0 -16583,0 -16584,0 -16585,0 -16586,0 -16587,0 -16588,0 -16589,0 -16590,0 -16591,0 -16592,0 -16593,0 -16594,0 -16595,0 -16596,0 -16597,0 -16598,0 -16599,0 -16600,0 -16601,0 -16602,0 -16603,0 -16604,0 -16605,0 -16606,0 -16607,0 -16608,0 -16609,0 -16610,0 -16611,0 -16612,0 -16613,0 -16614,0 -16615,0 -16616,0 -16617,0 -16618,0 -16619,0 -16620,0 -16621,0 -16622,0 -16623,0 -16624,0 -16625,0 -16626,0 -16627,0 -16628,0 -16629,0 -16630,0 -16631,0 -16632,0 -16633,0 -16634,0 -16635,0 -16636,0 -16637,0 -16638,0 -16639,0 -16640,0 -16641,0 -16642,0 -16643,0 -16644,0 -16645,0 -16646,0 -16647,0 -16648,0 -16649,0 -16650,0 -16651,0 -16652,0 -16653,0 -16654,0 -16655,0 -16656,0 -16657,0 -16658,0 -16659,0 -16660,0 -16661,0 -16662,0 -16663,0 -16664,0 -16665,0 -16666,0 -16667,0 -16668,0 -16669,0 -16670,0 -16671,0 -16672,0 -16673,0 -16674,0 -16675,0 -16676,0 -16677,0 -16678,0 -16679,0 -16680,0 -16681,0 -16682,0 -16683,0 -16684,0 -16685,0 -16686,0 -16687,0 -16688,0 -16689,0 -16690,0 -16691,0 -16692,0 -16693,0 -16694,0 -16695,0 -16696,0 -16697,0 -16698,0 -16699,0 -16700,0 -16701,0 -16702,0 -16703,0 -16704,0 -16705,0 -16706,0 -16707,0 -16708,0 -16709,0 -16710,0 -16711,0 -16712,0 -16713,0 -16714,0 -16715,0 -16716,0 -16717,0 -16718,0 -16719,0 -16720,0 -16721,0 -16722,0 -16723,0 -16724,0 -16725,0 -16726,0 -16727,0 -16728,0 -16729,0 -16730,0 -16731,0 -16732,0 -16733,0 -16734,0 -16735,0 -16736,0 -16737,0 -16738,0 -16739,0 -16740,0 -16741,0 -16742,0 -16743,0 -16744,0 -16745,0 -16746,0 -16747,0 -16748,0 -16749,0 -16750,0 -16751,0 -16752,0 -16753,0 -16754,0 -16755,0 -16756,0 -16757,0 -16758,0 -16759,0 -16760,0 -16761,0 -16762,0 -16763,0 -16764,0 -16765,0 -16766,0 -16767,0 -16768,0 -16769,0 -16770,0 -16771,0 -16772,0 -16773,0 -16774,0 -16775,0 -16776,0 -16777,0 -16778,0 -16779,0 -16780,0 -16781,0 -16782,0 -16783,0 -16784,0 -16785,0 -16786,0 -16787,0 -16788,0 -16789,0 -16790,0 -16791,0 -16792,0 -16793,0 -16794,0 -16795,0 -16796,0 -16797,0 -16798,0 -16799,0 -16800,0 -16801,0 -16802,0 -16803,0 -16804,0 -16805,0 -16806,0 -16807,0 -16808,0 -16809,0 -16810,0 -16811,0 -16812,0 -16813,0 -16814,0 -16815,0 -16816,0 -16817,0 -16818,0 -16819,0 -16820,0 -16821,0 -16822,0 -16823,0 -16824,0 -16825,0 -16826,0 -16827,0 -16828,0 -16829,0 -16830,0 -16831,0 -16832,0 -16833,0 -16834,0 -16835,0 -16836,0 -16837,0 -16838,0 -16839,0 -16840,0 -16841,0 -16842,0 -16843,0 -16844,0 -16845,0 -16846,0 -16847,0 -16848,0 -16849,0 -16850,0 -16851,0 -16852,0 -16853,0 -16854,0 -16855,0 -16856,0 -16857,0 -16858,0 -16859,0 -16860,0 -16861,0 -16862,0 -16863,0 -16864,0 -16865,0 -16866,0 -16867,0 -16868,0 -16869,0 -16870,0 -16871,0 -16872,0 -16873,0 -16874,0 -16875,0 -16876,0 -16877,0 -16878,0 -16879,0 -16880,0 -16881,0 -16882,0 -16883,0 -16884,0 -16885,0 -16886,0 -16887,0 -16888,0 -16889,0 -16890,0 -16891,0 -16892,0 -16893,0 -16894,0 -16895,0 -16896,0 -16897,0 -16898,0 -16899,0 -16900,0 -16901,0 -16902,0 -16903,0 -16904,0 -16905,0 -16906,0 -16907,0 -16908,0 -16909,0 -16910,0 -16911,0 -16912,0 -16913,0 -16914,0 -16915,0 -16916,0 -16917,0 -16918,0 -16919,0 -16920,0 -16921,0 -16922,0 -16923,0 -16924,0 -16925,0 -16926,0 -16927,0 -16928,0 -16929,0 -16930,0 -16931,0 -16932,0 -16933,0 -16934,0 -16935,0 -16936,0 -16937,0 -16938,0 -16939,0 -16940,0 -16941,0 -16942,0 -16943,0 -16944,0 -16945,0 -16946,0 -16947,0 -16948,0 -16949,0 -16950,0 -16951,0 -16952,0 -16953,0 -16954,0 -16955,0 -16956,0 -16957,0 -16958,0 -16959,0 -16960,0 -16961,0 -16962,0 -16963,0 -16964,0 -16965,0 -16966,0 -16967,0 -16968,0 -16969,0 -16970,0 -16971,0 -16972,0 -16973,0 -16974,0 -16975,0 -16976,0 -16977,0 -16978,0 -16979,0 -16980,0 -16981,0 -16982,0 -16983,0 -16984,0 -16985,0 -16986,0 -16987,0 -16988,0 -16989,0 -16990,0 -16991,0 -16992,0 -16993,0 -16994,0 -16995,0 -16996,0 -16997,0 -16998,0 -16999,0 -17000,0 -17001,0 -17002,0 -17003,0 -17004,0 -17005,0 -17006,0 -17007,0 -17008,0 -17009,0 -17010,0 -17011,0 -17012,0 -17013,0 -17014,0 -17015,0 -17016,0 -17017,0 -17018,0 -17019,0 -17020,0 -17021,0 -17022,0 -17023,0 -17024,0 -17025,0 -17026,0 -17027,0 -17028,0 -17029,0 -17030,0 -17031,0 -17032,0 -17033,0 -17034,0 -17035,0 -17036,0 -17037,0 -17038,0 -17039,0 -17040,0 -17041,0 -17042,0 -17043,0 -17044,0 -17045,0 -17046,0 -17047,0 -17048,0 -17049,0 -17050,0 -17051,0 -17052,0 -17053,0 -17054,0 -17055,0 -17056,0 -17057,0 -17058,0 -17059,0 -17060,0 -17061,0 -17062,0 -17063,0 -17064,0 -17065,0 -17066,0 -17067,0 -17068,0 -17069,0 -17070,0 -17071,0 -17072,0 -17073,0 -17074,0 -17075,0 -17076,0 -17077,0 -17078,0 -17079,0 -17080,0 -17081,0 -17082,0 -17083,0 -17084,0 -17085,0 -17086,0 -17087,0 -17088,0 -17089,0 -17090,0 -17091,0 -17092,0 -17093,0 -17094,0 -17095,0 -17096,0 -17097,0 -17098,0 -17099,0 -17100,0 -17101,0 -17102,0 -17103,0 -17104,0 -17105,0 -17106,0 -17107,0 -17108,0 -17109,0 -17110,0 -17111,0 -17112,0 -17113,0 -17114,0 -17115,0 -17116,0 -17117,0 -17118,0 -17119,0 -17120,0 -17121,0 -17122,0 -17123,0 -17124,0 -17125,0 -17126,0 -17127,0 -17128,0 -17129,0 -17130,0 -17131,0 -17132,0 -17133,0 -17134,0 -17135,0 -17136,0 -17137,0 -17138,0 -17139,0 -17140,0 -17141,0 -17142,0 -17143,0 -17144,0 -17145,0 -17146,0 -17147,0 -17148,0 -17149,0 -17150,0 -17151,0 -17152,0 -17153,0 -17154,0 -17155,0 -17156,0 -17157,0 -17158,0 -17159,0 -17160,0 -17161,0 -17162,0 -17163,0 -17164,0 -17165,0 -17166,0 -17167,0 -17168,0 -17169,0 -17170,0 -17171,0 -17172,0 -17173,0 -17174,0 -17175,0 -17176,0 -17177,0 -17178,0 -17179,0 -17180,0 -17181,0 -17182,0 -17183,0 -17184,0 -17185,0 -17186,0 -17187,0 -17188,0 -17189,0 -17190,0 -17191,0 -17192,0 -17193,0 -17194,0 -17195,0 -17196,0 -17197,0 -17198,0 -17199,0 -17200,0 -17201,0 -17202,0 -17203,0 -17204,0 -17205,0 -17206,0 -17207,0 -17208,0 -17209,0 -17210,0 -17211,0 -17212,0 -17213,0 -17214,0 -17215,0 -17216,0 -17217,0 -17218,0 -17219,0 -17220,0 -17221,0 -17222,0 -17223,0 -17224,0 -17225,0 -17226,0 -17227,0 -17228,0 -17229,0 -17230,0 -17231,0 -17232,0 -17233,0 -17234,0 -17235,0 -17236,0 -17237,0 -17238,0 -17239,0 -17240,0 -17241,0 -17242,0 -17243,0 -17244,0 -17245,0 -17246,0 -17247,0 -17248,0 -17249,0 -17250,0 -17251,0 -17252,0 -17253,0 -17254,0 -17255,0 -17256,0 -17257,0 -17258,0 -17259,0 -17260,0 -17261,0 -17262,0 -17263,0 -17264,0 -17265,0 -17266,0 -17267,0 -17268,0 -17269,0 -17270,0 -17271,0 -17272,0 -17273,0 -17274,0 -17275,0 -17276,0 -17277,0 -17278,0 -17279,0 -17280,0 -17281,0 -17282,0 -17283,0 -17284,0 -17285,0 -17286,0 -17287,0 -17288,0 -17289,0 -17290,0 -17291,0 -17292,0 -17293,0 -17294,0 -17295,0 -17296,0 -17297,0 -17298,0 -17299,0 -17300,0 -17301,0 -17302,0 -17303,0 -17304,0 -17305,0 -17306,0 -17307,0 -17308,0 -17309,0 -17310,0 -17311,0 -17312,0 -17313,0 -17314,0 -17315,0 -17316,0 -17317,0 -17318,0 -17319,0 -17320,0 -17321,0 -17322,0 -17323,0 -17324,0 -17325,0 -17326,0 -17327,0 -17328,0 -17329,0 -17330,0 -17331,0 -17332,0 -17333,0 -17334,0 -17335,0 -17336,0 -17337,0 -17338,0 -17339,0 -17340,0 -17341,0 -17342,0 -17343,0 -17344,0 -17345,0 -17346,0 -17347,0 -17348,0 -17349,0 -17350,0 -17351,0 -17352,0 -17353,0 -17354,0 -17355,0 -17356,0 -17357,0 -17358,0 -17359,0 -17360,0 -17361,0 -17362,0 -17363,0 -17364,0 -17365,0 -17366,0 -17367,0 -17368,0 -17369,0 -17370,0 -17371,0 -17372,0 -17373,0 -17374,0 -17375,0 -17376,0 -17377,0 -17378,0 -17379,0 -17380,0 -17381,0 -17382,0 -17383,0 -17384,0 -17385,0 -17386,0 -17387,0 -17388,0 -17389,0 -17390,0 -17391,0 -17392,0 -17393,0 -17394,0 -17395,0 -17396,0 -17397,0 -17398,0 -17399,0 -17400,0 -17401,0 -17402,0 -17403,0 -17404,0 -17405,0 -17406,0 -17407,0 -17408,0 -17409,0 -17410,0 -17411,0 -17412,0 -17413,0 -17414,0 -17415,0 -17416,0 -17417,0 -17418,0 -17419,0 -17420,0 -17421,0 -17422,0 -17423,0 -17424,0 -17425,0 -17426,0 -17427,0 -17428,0 -17429,0 -17430,0 -17431,0 -17432,0 -17433,0 -17434,0 -17435,0 -17436,0 -17437,0 -17438,0 -17439,0 -17440,0 -17441,0 -17442,0 -17443,0 -17444,0 -17445,0 -17446,0 -17447,0 -17448,0 -17449,0 -17450,0 -17451,0 -17452,0 -17453,0 -17454,0 -17455,0 -17456,0 -17457,0 -17458,0 -17459,0 -17460,0 -17461,0 -17462,0 -17463,0 -17464,0 -17465,0 -17466,0 -17467,0 -17468,0 -17469,0 -17470,0 -17471,0 -17472,0 -17473,0 -17474,0 -17475,0 -17476,0 -17477,0 -17478,0 -17479,0 -17480,0 -17481,0 -17482,0 -17483,0 -17484,0 -17485,0 -17486,0 -17487,0 -17488,0 -17489,0 -17490,0 -17491,0 -17492,0 -17493,0 -17494,0 -17495,0 -17496,0 -17497,0 -17498,0 -17499,0 -17500,0 -17501,0 -17502,0 -17503,0 -17504,0 -17505,0 -17506,0 -17507,0 -17508,0 -17509,0 -17510,0 -17511,0 -17512,0 -17513,0 -17514,0 -17515,0 -17516,0 -17517,0 -17518,0 -17519,0 -17520,0 -17521,0 -17522,0 -17523,0 -17524,0 -17525,0 -17526,0 -17527,0 -17528,0 -17529,0 -17530,0 -17531,0 -17532,0 -17533,0 -17534,0 -17535,0 -17536,0 -17537,0 -17538,0 -17539,0 -17540,0 -17541,0 -17542,0 -17543,0 -17544,0 -17545,0 -17546,0 -17547,0 -17548,0 -17549,0 -17550,0 -17551,0 -17552,0 -17553,0 -17554,0 -17555,0 -17556,0 -17557,0 -17558,0 -17559,0 -17560,0 -17561,0 -17562,0 -17563,0 -17564,0 -17565,0 -17566,0 -17567,0 -17568,0 -17569,0 -17570,0 -17571,0 -17572,0 -17573,0 -17574,0 -17575,0 -17576,0 -17577,0 -17578,0 -17579,0 -17580,0 -17581,0 -17582,0 -17583,0 -17584,0 -17585,0 -17586,0 -17587,0 -17588,0 -17589,0 -17590,0 -17591,0 -17592,0 -17593,0 -17594,0 -17595,0 -17596,0 -17597,0 -17598,0 -17599,0 -17600,0 -17601,0 -17602,0 -17603,0 -17604,0 -17605,0 -17606,0 -17607,0 -17608,0 -17609,0 -17610,0 -17611,0 -17612,0 -17613,0 -17614,0 -17615,0 -17616,0 -17617,0 -17618,0 -17619,0 -17620,0 -17621,0 -17622,0 -17623,0 -17624,0 -17625,0 -17626,0 -17627,0 -17628,0 -17629,0 -17630,0 -17631,0 -17632,0 -17633,0 -17634,0 -17635,0 -17636,0 -17637,0 -17638,0 -17639,0 -17640,0 -17641,0 -17642,0 -17643,0 -17644,0 -17645,0 -17646,0 -17647,0 -17648,0 -17649,0 -17650,0 -17651,0 -17652,0 -17653,0 -17654,0 -17655,0 -17656,0 -17657,0 -17658,0 -17659,0 -17660,0 -17661,0 -17662,0 -17663,0 -17664,0 -17665,0 -17666,0 -17667,0 -17668,0 -17669,0 -17670,0 -17671,0 -17672,0 -17673,0 -17674,0 -17675,0 -17676,0 -17677,0 -17678,0 -17679,0 -17680,0 -17681,0 -17682,0 -17683,0 -17684,0 -17685,0 -17686,0 -17687,0 -17688,0 -17689,0 -17690,0 -17691,0 -17692,0 -17693,0 -17694,0 -17695,0 -17696,0 -17697,0 -17698,0 -17699,0 -17700,0 -17701,0 -17702,0 -17703,0 -17704,0 -17705,0 -17706,0 -17707,0 -17708,0 -17709,0 -17710,0 -17711,0 -17712,0 -17713,0 -17714,0 -17715,0 -17716,0 -17717,0 -17718,0 -17719,0 -17720,0 -17721,0 -17722,0 -17723,0 -17724,0 -17725,0 -17726,0 -17727,0 -17728,0 -17729,0 -17730,0 -17731,0 -17732,0 -17733,0 -17734,0 -17735,0 -17736,0 -17737,0 -17738,0 -17739,0 -17740,0 -17741,0 -17742,0 -17743,0 -17744,0 -17745,0 -17746,0 -17747,0 -17748,0 -17749,0 -17750,0 -17751,0 -17752,0 -17753,0 -17754,0 -17755,0 -17756,0 -17757,0 -17758,0 -17759,0 -17760,0 -17761,0 -17762,0 -17763,0 -17764,0 -17765,0 -17766,0 -17767,0 -17768,0 -17769,0 -17770,0 -17771,0 -17772,0 -17773,0 -17774,0 -17775,0 -17776,0 -17777,0 -17778,0 -17779,0 -17780,0 -17781,0 -17782,0 -17783,0 -17784,0 -17785,0 -17786,0 -17787,0 -17788,0 -17789,0 -17790,0 -17791,0 -17792,0 -17793,0 -17794,0 -17795,0 -17796,0 -17797,0 -17798,0 -17799,0 -17800,0 -17801,0 -17802,0 -17803,0 -17804,0 -17805,0 -17806,0 -17807,0 -17808,0 -17809,0 -17810,0 -17811,0 -17812,0 -17813,0 -17814,0 -17815,0 -17816,0 -17817,0 -17818,0 -17819,0 -17820,0 -17821,0 -17822,0 -17823,0 -17824,0 -17825,0 -17826,0 -17827,0 -17828,0 -17829,0 -17830,0 -17831,0 -17832,0 -17833,0 -17834,0 -17835,0 -17836,0 -17837,0 -17838,0 -17839,0 -17840,0 -17841,0 -17842,0 -17843,0 -17844,0 -17845,0 -17846,0 -17847,0 -17848,0 -17849,0 -17850,0 -17851,0 -17852,0 -17853,0 -17854,0 -17855,0 -17856,0 -17857,0 -17858,0 -17859,0 -17860,0 -17861,0 -17862,0 -17863,0 -17864,0 -17865,0 -17866,0 -17867,0 -17868,0 -17869,0 -17870,0 -17871,0 -17872,0 -17873,0 -17874,0 -17875,0 -17876,0 -17877,0 -17878,0 -17879,0 -17880,0 -17881,0 -17882,0 -17883,0 -17884,0 -17885,0 -17886,0 -17887,0 -17888,0 -17889,0 -17890,0 -17891,0 -17892,0 -17893,0 -17894,0 -17895,0 -17896,0 -17897,0 -17898,0 -17899,0 -17900,0 -17901,0 -17902,0 -17903,0 -17904,0 -17905,0 -17906,0 -17907,0 -17908,0 -17909,0 -17910,0 -17911,0 -17912,0 -17913,0 -17914,0 -17915,0 -17916,0 -17917,0 -17918,0 -17919,0 -17920,0 -17921,0 -17922,0 -17923,0 -17924,0 -17925,0 -17926,0 -17927,0 -17928,0 -17929,0 -17930,0 -17931,0 -17932,0 -17933,0 -17934,0 -17935,0 -17936,0 -17937,0 -17938,0 -17939,0 -17940,0 -17941,0 -17942,0 -17943,0 -17944,0 -17945,0 -17946,0 -17947,0 -17948,0 -17949,0 -17950,0 -17951,0 -17952,0 -17953,0 -17954,0 -17955,0 -17956,0 -17957,0 -17958,0 -17959,0 -17960,0 -17961,0 -17962,0 -17963,0 -17964,0 -17965,0 -17966,0 -17967,0 -17968,0 -17969,0 -17970,0 -17971,0 -17972,0 -17973,0 -17974,0 -17975,0 -17976,0 -17977,0 -17978,0 -17979,0 -17980,0 -17981,0 -17982,0 -17983,0 -17984,0 -17985,0 -17986,0 -17987,0 -17988,0 -17989,0 -17990,0 -17991,0 -17992,0 -17993,0 -17994,0 -17995,0 -17996,0 -17997,0 -17998,0 -17999,0 -18000,0 -18001,0 -18002,0 -18003,0 -18004,0 -18005,0 -18006,0 -18007,0 -18008,0 -18009,0 -18010,0 -18011,0 -18012,0 -18013,0 -18014,0 -18015,0 -18016,0 -18017,0 -18018,0 -18019,0 -18020,0 -18021,0 -18022,0 -18023,0 -18024,0 -18025,0 -18026,0 -18027,0 -18028,0 -18029,0 -18030,0 -18031,0 -18032,0 -18033,0 -18034,0 -18035,0 -18036,0 -18037,0 -18038,0 -18039,0 -18040,0 -18041,0 -18042,0 -18043,0 -18044,0 -18045,0 -18046,0 -18047,0 -18048,0 -18049,0 -18050,0 -18051,0 -18052,0 -18053,0 -18054,0 -18055,0 -18056,0 -18057,0 -18058,0 -18059,0 -18060,0 -18061,0 -18062,0 -18063,0 -18064,0 -18065,0 -18066,0 -18067,0 -18068,0 -18069,0 -18070,0 -18071,0 -18072,0 -18073,0 -18074,0 -18075,0 -18076,0 -18077,0 -18078,0 -18079,0 -18080,0 -18081,0 -18082,0 -18083,0 -18084,0 -18085,0 -18086,0 -18087,0 -18088,0 -18089,0 -18090,0 -18091,0 -18092,0 -18093,0 -18094,0 -18095,0 -18096,0 -18097,0 -18098,0 -18099,0 -18100,0 -18101,0 -18102,0 -18103,0 -18104,0 -18105,0 -18106,0 -18107,0 -18108,0 -18109,0 -18110,0 -18111,0 -18112,0 -18113,0 -18114,0 -18115,0 -18116,0 -18117,0 -18118,0 -18119,0 -18120,0 -18121,0 -18122,0 -18123,0 -18124,0 -18125,0 -18126,0 -18127,0 -18128,0 -18129,0 -18130,0 -18131,0 -18132,0 -18133,0 -18134,0 -18135,0 -18136,0 -18137,0 -18138,0 -18139,0 -18140,0 -18141,0 -18142,0 -18143,0 -18144,0 -18145,0 -18146,0 -18147,0 -18148,0 -18149,0 -18150,0 -18151,0 -18152,0 -18153,0 -18154,0 -18155,0 -18156,0 -18157,0 -18158,0 -18159,0 -18160,0 -18161,0 -18162,0 -18163,0 -18164,0 -18165,0 -18166,0 -18167,0 -18168,0 -18169,0 -18170,0 -18171,0 -18172,0 -18173,0 -18174,0 -18175,0 -18176,0 -18177,0 -18178,0 -18179,0 -18180,0 -18181,0 -18182,0 -18183,0 -18184,0 -18185,0 -18186,0 -18187,0 -18188,0 -18189,0 -18190,0 -18191,0 -18192,0 -18193,0 -18194,0 -18195,0 -18196,0 -18197,0 -18198,0 -18199,0 -18200,0 -18201,0 -18202,0 -18203,0 -18204,0 -18205,0 -18206,0 -18207,0 -18208,0 -18209,0 -18210,0 -18211,0 -18212,0 -18213,0 -18214,0 -18215,0 -18216,0 -18217,0 -18218,0 -18219,0 -18220,0 -18221,0 -18222,0 -18223,0 -18224,0 -18225,0 -18226,0 -18227,0 -18228,0 -18229,0 -18230,0 -18231,0 -18232,0 -18233,0 -18234,0 -18235,0 -18236,0 -18237,0 -18238,0 -18239,0 -18240,0 -18241,0 -18242,0 -18243,0 -18244,0 -18245,0 -18246,0 -18247,0 -18248,0 -18249,0 -18250,0 -18251,0 -18252,0 -18253,0 -18254,0 -18255,0 -18256,0 -18257,0 -18258,0 -18259,0 -18260,0 -18261,0 -18262,0 -18263,0 -18264,0 -18265,0 -18266,0 -18267,0 -18268,0 -18269,0 -18270,0 -18271,0 -18272,0 -18273,0 -18274,0 -18275,0 -18276,0 -18277,0 -18278,0 -18279,0 -18280,0 -18281,0 -18282,0 -18283,0 -18284,0 -18285,0 -18286,0 -18287,0 -18288,0 -18289,0 -18290,0 -18291,0 -18292,0 -18293,0 -18294,0 -18295,0 -18296,0 -18297,0 -18298,0 -18299,0 -18300,0 -18301,0 -18302,0 -18303,0 -18304,0 -18305,0 -18306,0 -18307,0 -18308,0 -18309,0 -18310,0 -18311,0 -18312,0 -18313,0 -18314,0 -18315,0 -18316,0 -18317,0 -18318,0 -18319,0 -18320,0 -18321,0 -18322,0 -18323,0 -18324,0 -18325,0 -18326,0 -18327,0 -18328,0 -18329,0 -18330,0 -18331,0 -18332,0 -18333,0 -18334,0 -18335,0 -18336,0 -18337,0 -18338,0 -18339,0 -18340,0 -18341,0 -18342,0 -18343,0 -18344,0 -18345,0 -18346,0 -18347,0 -18348,0 -18349,0 -18350,0 -18351,0 -18352,0 -18353,0 -18354,0 -18355,0 -18356,0 -18357,0 -18358,0 -18359,0 -18360,0 -18361,0 -18362,0 -18363,0 -18364,0 -18365,0 -18366,0 -18367,0 -18368,0 -18369,0 -18370,0 -18371,0 -18372,0 -18373,0 -18374,0 -18375,0 -18376,0 -18377,0 -18378,0 -18379,0 -18380,0 -18381,0 -18382,0 -18383,0 -18384,0 -18385,0 -18386,0 -18387,0 -18388,0 -18389,0 -18390,0 -18391,0 -18392,0 -18393,0 -18394,0 -18395,0 -18396,0 -18397,0 -18398,0 -18399,0 -18400,0 -18401,0 -18402,0 -18403,0 -18404,0 -18405,0 -18406,0 -18407,0 -18408,0 -18409,0 -18410,0 -18411,0 -18412,0 -18413,0 -18414,0 -18415,0 -18416,0 -18417,0 -18418,0 -18419,0 -18420,0 -18421,0 -18422,0 -18423,0 -18424,0 -18425,0 -18426,0 -18427,0 -18428,0 -18429,0 -18430,0 -18431,0 -18432,0 -18433,0 -18434,0 -18435,0 -18436,0 -18437,0 -18438,0 -18439,0 -18440,0 -18441,0 -18442,0 -18443,0 -18444,0 -18445,0 -18446,0 -18447,0 -18448,0 -18449,0 -18450,0 -18451,0 -18452,0 -18453,0 -18454,0 -18455,0 -18456,0 -18457,0 -18458,0 -18459,0 -18460,0 -18461,0 -18462,0 -18463,0 -18464,0 -18465,0 -18466,0 -18467,0 -18468,0 -18469,0 -18470,0 -18471,0 -18472,0 -18473,0 -18474,0 -18475,0 -18476,0 -18477,0 -18478,0 -18479,0 -18480,0 -18481,0 -18482,0 -18483,0 -18484,0 -18485,0 -18486,0 -18487,0 -18488,0 -18489,0 -18490,0 -18491,0 -18492,0 -18493,0 -18494,0 -18495,0 -18496,0 -18497,0 -18498,0 -18499,0 -18500,0 -18501,0 -18502,0 -18503,0 -18504,0 -18505,0 -18506,0 -18507,0 -18508,0 -18509,0 -18510,0 -18511,0 -18512,0 -18513,0 -18514,0 -18515,0 -18516,0 -18517,0 -18518,0 -18519,0 -18520,0 -18521,0 -18522,0 -18523,0 -18524,0 -18525,0 -18526,0 -18527,0 -18528,0 -18529,0 -18530,0 -18531,0 -18532,0 -18533,0 -18534,0 -18535,0 -18536,0 -18537,0 -18538,0 -18539,0 -18540,0 -18541,0 -18542,0 -18543,0 -18544,0 -18545,0 -18546,0 -18547,0 -18548,0 -18549,0 -18550,0 -18551,0 -18552,0 -18553,0 -18554,0 -18555,0 -18556,0 -18557,0 -18558,0 -18559,0 -18560,0 -18561,0 -18562,0 -18563,0 -18564,0 -18565,0 -18566,0 -18567,0 -18568,0 -18569,0 -18570,0 -18571,0 -18572,0 -18573,0 -18574,0 -18575,0 -18576,0 -18577,0 -18578,0 -18579,0 -18580,0 -18581,0 -18582,0 -18583,0 -18584,0 -18585,0 -18586,0 -18587,0 -18588,0 -18589,0 -18590,0 -18591,0 -18592,0 -18593,0 -18594,0 -18595,0 -18596,0 -18597,0 -18598,0 -18599,0 -18600,0 -18601,0 -18602,0 -18603,0 -18604,0 -18605,0 -18606,0 -18607,0 -18608,0 -18609,0 -18610,0 -18611,0 -18612,0 -18613,0 -18614,0 -18615,0 -18616,0 -18617,0 -18618,0 -18619,0 -18620,0 -18621,0 -18622,0 -18623,0 -18624,0 -18625,0 -18626,0 -18627,0 -18628,0 -18629,0 -18630,0 -18631,0 -18632,0 -18633,0 -18634,0 -18635,0 -18636,0 -18637,0 -18638,0 -18639,0 -18640,0 -18641,0 -18642,0 -18643,0 -18644,0 -18645,0 -18646,0 -18647,0 -18648,0 -18649,0 -18650,0 -18651,0 -18652,0 -18653,0 -18654,0 -18655,0 -18656,0 -18657,0 -18658,0 -18659,0 -18660,0 -18661,0 -18662,0 -18663,0 -18664,0 -18665,0 -18666,0 -18667,0 -18668,0 -18669,0 -18670,0 -18671,0 -18672,0 -18673,0 -18674,0 -18675,0 -18676,0 -18677,0 -18678,0 -18679,0 -18680,0 -18681,0 -18682,0 -18683,0 -18684,0 -18685,0 -18686,0 -18687,0 -18688,0 -18689,0 -18690,0 -18691,0 -18692,0 -18693,0 -18694,0 -18695,0 -18696,0 -18697,0 -18698,0 -18699,0 -18700,0 -18701,0 -18702,0 -18703,0 -18704,0 -18705,0 -18706,0 -18707,0 -18708,0 -18709,0 -18710,0 -18711,0 -18712,0 -18713,0 -18714,0 -18715,0 -18716,0 -18717,0 -18718,0 -18719,0 -18720,0 -18721,0 -18722,0 -18723,0 -18724,0 -18725,0 -18726,0 -18727,0 -18728,0 -18729,0 -18730,0 -18731,0 -18732,0 -18733,0 -18734,0 -18735,0 -18736,0 -18737,0 -18738,0 -18739,0 -18740,0 -18741,0 -18742,0 -18743,0 -18744,0 -18745,0 -18746,0 -18747,0 -18748,0 -18749,0 -18750,0 -18751,0 -18752,0 -18753,0 -18754,0 -18755,0 -18756,0 -18757,0 -18758,0 -18759,0 -18760,0 -18761,0 -18762,0 -18763,0 -18764,0 -18765,0 -18766,0 -18767,0 -18768,0 -18769,0 -18770,0 -18771,0 -18772,0 -18773,0 -18774,0 -18775,0 -18776,0 -18777,0 -18778,0 -18779,0 -18780,0 -18781,0 -18782,0 -18783,0 -18784,0 -18785,0 -18786,0 -18787,0 -18788,0 -18789,0 -18790,0 -18791,0 -18792,0 -18793,0 -18794,0 -18795,0 -18796,0 -18797,0 -18798,0 -18799,0 -18800,0 -18801,0 -18802,0 -18803,0 -18804,0 -18805,0 -18806,0 -18807,0 -18808,0 -18809,0 -18810,0 -18811,0 -18812,0 -18813,0 -18814,0 -18815,0 -18816,0 -18817,0 -18818,0 -18819,0 -18820,0 -18821,0 -18822,0 -18823,0 -18824,0 -18825,0 -18826,0 -18827,0 -18828,0 -18829,0 -18830,0 -18831,0 -18832,0 -18833,0 -18834,0 -18835,0 -18836,0 -18837,0 -18838,0 -18839,0 -18840,0 -18841,0 -18842,0 -18843,0 -18844,0 -18845,0 -18846,0 -18847,0 -18848,0 -18849,0 -18850,0 -18851,0 -18852,0 -18853,0 -18854,0 -18855,0 -18856,0 -18857,0 -18858,0 -18859,0 -18860,0 -18861,0 -18862,0 -18863,0 -18864,0 -18865,0 -18866,0 -18867,0 -18868,0 -18869,0 -18870,0 -18871,0 -18872,0 -18873,0 -18874,0 -18875,0 -18876,0 -18877,0 -18878,0 -18879,0 -18880,0 -18881,0 -18882,0 -18883,0 -18884,0 -18885,0 -18886,0 -18887,0 -18888,0 -18889,0 -18890,0 -18891,0 -18892,0 -18893,0 -18894,0 -18895,0 -18896,0 -18897,0 -18898,0 -18899,0 -18900,0 -18901,0 -18902,0 -18903,0 -18904,0 -18905,0 -18906,0 -18907,0 -18908,0 -18909,0 -18910,0 -18911,0 -18912,0 -18913,0 -18914,0 -18915,0 -18916,0 -18917,0 -18918,0 -18919,0 -18920,0 -18921,0 -18922,0 -18923,0 -18924,0 -18925,0 -18926,0 -18927,0 -18928,0 -18929,0 -18930,0 -18931,0 -18932,0 -18933,0 -18934,0 -18935,0 -18936,0 -18937,0 -18938,0 -18939,0 -18940,0 -18941,0 -18942,0 -18943,0 -18944,0 -18945,0 -18946,0 -18947,0 -18948,0 -18949,0 -18950,0 -18951,0 -18952,0 -18953,0 -18954,0 -18955,0 -18956,0 -18957,0 -18958,0 -18959,0 -18960,0 -18961,0 -18962,0 -18963,0 -18964,0 -18965,0 -18966,0 -18967,0 -18968,0 -18969,0 -18970,0 -18971,0 -18972,0 -18973,0 -18974,0 -18975,0 -18976,0 -18977,0 -18978,0 -18979,0 -18980,0 -18981,0 -18982,0 -18983,0 -18984,0 -18985,0 -18986,0 -18987,0 -18988,0 -18989,0 -18990,0 -18991,0 -18992,0 -18993,0 -18994,0 -18995,0 -18996,0 -18997,0 -18998,0 -18999,0 -19000,0 -19001,0 -19002,0 -19003,0 -19004,0 -19005,0 -19006,0 -19007,0 -19008,0 -19009,0 -19010,0 -19011,0 -19012,0 -19013,0 -19014,0 -19015,0 -19016,0 -19017,0 -19018,0 -19019,0 -19020,0 -19021,0 -19022,0 -19023,0 -19024,0 -19025,0 -19026,0 -19027,0 -19028,0 -19029,0 -19030,0 -19031,0 -19032,0 -19033,0 -19034,0 -19035,0 -19036,0 -19037,0 -19038,0 -19039,0 -19040,0 -19041,0 -19042,0 -19043,0 -19044,0 -19045,0 -19046,0 -19047,0 -19048,0 -19049,0 -19050,0 -19051,0 -19052,0 -19053,0 -19054,0 -19055,0 -19056,0 -19057,0 -19058,0 -19059,0 -19060,0 -19061,0 -19062,0 -19063,0 -19064,0 -19065,0 -19066,0 -19067,0 -19068,0 -19069,0 -19070,0 -19071,0 -19072,0 -19073,0 -19074,0 -19075,0 -19076,0 -19077,0 -19078,0 -19079,0 -19080,0 -19081,0 -19082,0 -19083,0 -19084,0 -19085,0 -19086,0 -19087,0 -19088,0 -19089,0 -19090,0 -19091,0 -19092,0 -19093,0 -19094,0 -19095,0 -19096,0 -19097,0 -19098,0 -19099,0 -19100,0 -19101,0 -19102,0 -19103,0 -19104,0 -19105,0 -19106,0 -19107,0 -19108,0 -19109,0 -19110,0 -19111,0 -19112,0 -19113,0 -19114,0 -19115,0 -19116,0 -19117,0 -19118,0 -19119,0 -19120,0 -19121,0 -19122,0 -19123,0 -19124,0 -19125,0 -19126,0 -19127,0 -19128,0 -19129,0 -19130,0 -19131,0 -19132,0 -19133,0 -19134,0 -19135,0 -19136,0 -19137,0 -19138,0 -19139,0 -19140,0 -19141,0 -19142,0 -19143,0 -19144,0 -19145,0 -19146,0 -19147,0 -19148,0 -19149,0 -19150,0 -19151,0 -19152,0 -19153,0 -19154,0 -19155,0 -19156,0 -19157,0 -19158,0 -19159,0 -19160,0 -19161,0 -19162,0 -19163,0 -19164,0 -19165,0 -19166,0 -19167,0 -19168,0 -19169,0 -19170,0 -19171,0 -19172,0 -19173,0 -19174,0 -19175,0 -19176,0 -19177,0 -19178,0 -19179,0 -19180,0 -19181,0 -19182,0 -19183,0 -19184,0 -19185,0 -19186,0 -19187,0 -19188,0 -19189,0 -19190,0 -19191,0 -19192,0 -19193,0 -19194,0 -19195,0 -19196,0 -19197,0 -19198,0 -19199,0 -19200,0 -19201,0 -19202,0 -19203,0 -19204,0 -19205,0 -19206,0 -19207,0 -19208,0 -19209,0 -19210,0 -19211,0 -19212,0 -19213,0 -19214,0 -19215,0 -19216,0 -19217,0 -19218,0 -19219,0 -19220,0 -19221,0 -19222,0 -19223,0 -19224,0 -19225,0 -19226,0 -19227,0 -19228,0 -19229,0 -19230,0 -19231,0 -19232,0 -19233,0 -19234,0 -19235,0 -19236,0 -19237,0 -19238,0 -19239,0 -19240,0 -19241,0 -19242,0 -19243,0 -19244,0 -19245,0 -19246,0 -19247,0 -19248,0 -19249,0 -19250,0 -19251,0 -19252,0 -19253,0 -19254,0 -19255,0 -19256,0 -19257,0 -19258,0 -19259,0 -19260,0 -19261,0 -19262,0 -19263,0 -19264,0 -19265,0 -19266,0 -19267,0 -19268,0 -19269,0 -19270,0 -19271,0 -19272,0 -19273,0 -19274,0 -19275,0 -19276,0 -19277,0 -19278,0 -19279,0 -19280,0 -19281,0 -19282,0 -19283,0 -19284,0 -19285,0 -19286,0 -19287,0 -19288,0 -19289,0 -19290,0 -19291,0 -19292,0 -19293,0 -19294,0 -19295,0 -19296,0 -19297,0 -19298,0 -19299,0 -19300,0 -19301,0 -19302,0 -19303,0 -19304,0 -19305,0 -19306,0 -19307,0 -19308,0 -19309,0 -19310,0 -19311,0 -19312,0 -19313,0 -19314,0 -19315,0 -19316,0 -19317,0 -19318,0 -19319,0 -19320,0 -19321,0 -19322,0 -19323,0 -19324,0 -19325,0 -19326,0 -19327,0 -19328,0 -19329,0 -19330,0 -19331,0 -19332,0 -19333,0 -19334,0 -19335,0 -19336,0 -19337,0 -19338,0 -19339,0 -19340,0 -19341,0 -19342,0 -19343,0 -19344,0 -19345,0 -19346,0 -19347,0 -19348,0 -19349,0 -19350,0 -19351,0 -19352,0 -19353,0 -19354,0 -19355,0 -19356,0 -19357,0 -19358,0 -19359,0 -19360,0 -19361,0 -19362,0 -19363,0 -19364,0 -19365,0 -19366,0 -19367,0 -19368,0 -19369,0 -19370,0 -19371,0 -19372,0 -19373,0 -19374,0 -19375,0 -19376,0 -19377,0 -19378,0 -19379,0 -19380,0 -19381,0 -19382,0 -19383,0 -19384,0 -19385,0 -19386,0 -19387,0 -19388,0 -19389,0 -19390,0 -19391,0 -19392,0 -19393,0 -19394,0 -19395,0 -19396,0 -19397,0 -19398,0 -19399,0 -19400,0 -19401,0 -19402,0 -19403,0 -19404,0 -19405,0 -19406,0 -19407,0 -19408,0 -19409,0 -19410,0 -19411,0 -19412,0 -19413,0 -19414,0 -19415,0 -19416,0 -19417,0 -19418,0 -19419,0 -19420,0 -19421,0 -19422,0 -19423,0 -19424,0 -19425,0 -19426,0 -19427,0 -19428,0 -19429,0 -19430,0 -19431,0 -19432,0 -19433,0 -19434,0 -19435,0 -19436,0 -19437,0 -19438,0 -19439,0 -19440,0 -19441,0 -19442,0 -19443,0 -19444,0 -19445,0 -19446,0 -19447,0 -19448,0 -19449,0 -19450,0 -19451,0 -19452,0 -19453,0 -19454,0 -19455,0 -19456,0 -19457,0 -19458,0 -19459,0 -19460,0 -19461,0 -19462,0 -19463,0 -19464,0 -19465,0 -19466,0 -19467,0 -19468,0 -19469,0 -19470,0 -19471,0 -19472,0 -19473,0 -19474,0 -19475,0 -19476,0 -19477,0 -19478,0 -19479,0 -19480,0 -19481,0 -19482,0 -19483,0 -19484,0 -19485,0 -19486,0 -19487,0 -19488,0 -19489,0 -19490,0 -19491,0 -19492,0 -19493,0 -19494,0 -19495,0 -19496,0 -19497,0 -19498,0 -19499,0 -19500,0 -19501,0 -19502,0 -19503,0 -19504,0 -19505,0 -19506,0 -19507,0 -19508,0 -19509,0 -19510,0 -19511,0 -19512,0 -19513,0 -19514,0 -19515,0 -19516,0 -19517,0 -19518,0 -19519,0 -19520,0 -19521,0 -19522,0 -19523,0 -19524,0 -19525,0 -19526,0 -19527,0 -19528,0 -19529,0 -19530,0 -19531,0 -19532,0 -19533,0 -19534,0 -19535,0 -19536,0 -19537,0 -19538,0 -19539,0 -19540,0 -19541,0 -19542,0 -19543,0 -19544,0 -19545,0 -19546,0 -19547,0 -19548,0 -19549,0 -19550,0 -19551,0 -19552,0 -19553,0 -19554,0 -19555,0 -19556,0 -19557,0 -19558,0 -19559,0 -19560,0 -19561,0 -19562,0 -19563,0 -19564,0 -19565,0 -19566,0 -19567,0 -19568,0 -19569,0 -19570,0 -19571,0 -19572,0 -19573,0 -19574,0 -19575,0 -19576,0 -19577,0 -19578,0 -19579,0 -19580,0 -19581,0 -19582,0 -19583,0 -19584,0 -19585,0 -19586,0 -19587,0 -19588,0 -19589,0 -19590,0 -19591,0 -19592,0 -19593,0 -19594,0 -19595,0 -19596,0 -19597,0 -19598,0 -19599,0 -19600,0 -19601,0 -19602,0 -19603,0 -19604,0 -19605,0 -19606,0 -19607,0 -19608,0 -19609,0 -19610,0 -19611,0 -19612,0 -19613,0 -19614,0 -19615,0 -19616,0 -19617,0 -19618,0 -19619,0 -19620,0 -19621,0 -19622,0 -19623,0 -19624,0 -19625,0 -19626,0 -19627,0 -19628,0 -19629,0 -19630,0 -19631,0 -19632,0 -19633,0 -19634,0 -19635,0 -19636,0 -19637,0 -19638,0 -19639,0 -19640,0 -19641,0 -19642,0 -19643,0 -19644,0 -19645,0 -19646,0 -19647,0 -19648,0 -19649,0 -19650,0 -19651,0 -19652,0 -19653,0 -19654,0 -19655,0 -19656,0 -19657,0 -19658,0 -19659,0 -19660,0 -19661,0 -19662,0 -19663,0 -19664,0 -19665,0 -19666,0 -19667,0 -19668,0 -19669,0 -19670,0 -19671,0 -19672,0 -19673,0 -19674,0 -19675,0 -19676,0 -19677,0 -19678,0 -19679,0 -19680,0 -19681,0 -19682,0 -19683,0 -19684,0 -19685,0 -19686,0 -19687,0 -19688,0 -19689,0 -19690,0 -19691,0 -19692,0 -19693,0 -19694,0 -19695,0 -19696,0 -19697,0 -19698,0 -19699,0 -19700,0 -19701,0 -19702,0 -19703,0 -19704,0 -19705,0 -19706,0 -19707,0 -19708,0 -19709,0 -19710,0 -19711,0 -19712,0 -19713,0 -19714,0 -19715,0 -19716,0 -19717,0 -19718,0 -19719,0 -19720,0 -19721,0 -19722,0 -19723,0 -19724,0 -19725,0 -19726,0 -19727,0 -19728,0 -19729,0 -19730,0 -19731,0 -19732,0 -19733,0 -19734,0 -19735,0 -19736,0 -19737,0 -19738,0 -19739,0 -19740,0 -19741,0 -19742,0 -19743,0 -19744,0 -19745,0 -19746,0 -19747,0 -19748,0 -19749,0 -19750,0 -19751,0 -19752,0 -19753,0 -19754,0 -19755,0 -19756,0 -19757,0 -19758,0 -19759,0 -19760,0 -19761,0 -19762,0 -19763,0 -19764,0 -19765,0 -19766,0 -19767,0 -19768,0 -19769,0 -19770,0 -19771,0 -19772,0 -19773,0 -19774,0 -19775,0 -19776,0 -19777,0 -19778,0 -19779,0 -19780,0 -19781,0 -19782,0 -19783,0 -19784,0 -19785,0 -19786,0 -19787,0 -19788,0 -19789,0 -19790,0 -19791,0 -19792,0 -19793,0 -19794,0 -19795,0 -19796,0 -19797,0 -19798,0 -19799,0 -19800,0 -19801,0 -19802,0 -19803,0 -19804,0 -19805,0 -19806,0 -19807,0 -19808,0 -19809,0 -19810,0 -19811,0 -19812,0 -19813,0 -19814,0 -19815,0 -19816,0 -19817,0 -19818,0 -19819,0 -19820,0 -19821,0 -19822,0 -19823,0 -19824,0 -19825,0 -19826,0 -19827,0 -19828,0 -19829,0 -19830,0 -19831,0 -19832,0 -19833,0 -19834,0 -19835,0 -19836,0 -19837,0 -19838,0 -19839,0 -19840,0 -19841,0 -19842,0 -19843,0 -19844,0 -19845,0 -19846,0 -19847,0 -19848,0 -19849,0 -19850,0 -19851,0 -19852,0 -19853,0 -19854,0 -19855,0 -19856,0 -19857,0 -19858,0 -19859,0 -19860,0 -19861,0 -19862,0 -19863,0 -19864,0 -19865,0 -19866,0 -19867,0 -19868,0 -19869,0 -19870,0 -19871,0 -19872,0 -19873,0 -19874,0 -19875,0 -19876,0 -19877,0 -19878,0 -19879,0 -19880,0 -19881,0 -19882,0 -19883,0 -19884,0 -19885,0 -19886,0 -19887,0 -19888,0 -19889,0 -19890,0 -19891,0 -19892,0 -19893,0 -19894,0 -19895,0 -19896,0 -19897,0 -19898,0 -19899,0 -19900,0 -19901,0 -19902,0 -19903,0 -19904,0 -19905,0 -19906,0 -19907,0 -19908,0 -19909,0 -19910,0 -19911,0 -19912,0 -19913,0 -19914,0 -19915,0 -19916,0 -19917,0 -19918,0 -19919,0 -19920,0 -19921,0 -19922,0 -19923,0 -19924,0 -19925,0 -19926,0 -19927,0 -19928,0 -19929,0 -19930,0 -19931,0 -19932,0 -19933,0 -19934,0 -19935,0 -19936,0 -19937,0 -19938,0 -19939,0 -19940,0 -19941,0 -19942,0 -19943,0 -19944,0 -19945,0 -19946,0 -19947,0 -19948,0 -19949,0 -19950,0 -19951,0 -19952,0 -19953,0 -19954,0 -19955,0 -19956,0 -19957,0 -19958,0 -19959,0 -19960,0 -19961,0 -19962,0 -19963,0 -19964,0 -19965,0 -19966,0 -19967,0 -19968,0 -19969,0 -19970,0 -19971,0 -19972,0 -19973,0 -19974,0 -19975,0 -19976,0 -19977,0 -19978,0 -19979,0 -19980,0 -19981,0 -19982,0 -19983,0 -19984,0 -19985,0 -19986,0 -19987,0 -19988,0 -19989,0 -19990,0 -19991,0 -19992,0 -19993,0 -19994,0 -19995,0 -19996,0 -19997,0 -19998,0 -19999,0 -20000,0 -20001,0 -20002,0 -20003,0 -20004,0 -20005,0 -20006,0 -20007,0 -20008,0 -20009,0 -20010,0 -20011,0 -20012,0 -20013,0 -20014,0 -20015,0 -20016,0 -20017,0 -20018,0 -20019,0 -20020,0 -20021,0 -20022,0 -20023,0 -20024,0 -20025,0 -20026,0 -20027,0 -20028,0 -20029,0 -20030,0 -20031,0 -20032,0 -20033,0 -20034,0 -20035,0 -20036,0 -20037,0 -20038,0 -20039,0 -20040,0 -20041,0 -20042,0 -20043,0 -20044,0 -20045,0 -20046,0 -20047,0 -20048,0 -20049,0 -20050,0 -20051,0 -20052,0 -20053,0 -20054,0 -20055,0 -20056,0 -20057,0 -20058,0 -20059,0 -20060,0 -20061,0 -20062,0 -20063,0 -20064,0 -20065,0 -20066,0 -20067,0 -20068,0 -20069,0 -20070,0 -20071,0 -20072,0 -20073,0 -20074,0 -20075,0 -20076,0 -20077,0 -20078,0 -20079,0 -20080,0 -20081,0 -20082,0 -20083,0 -20084,0 -20085,0 -20086,0 -20087,0 -20088,0 -20089,0 -20090,0 -20091,0 -20092,0 -20093,0 -20094,0 -20095,0 -20096,0 -20097,0 -20098,0 -20099,0 -20100,0 -20101,0 -20102,0 -20103,0 -20104,0 -20105,0 -20106,0 -20107,0 -20108,0 -20109,0 -20110,0 -20111,0 -20112,0 -20113,0 -20114,0 -20115,0 -20116,0 -20117,0 -20118,0 -20119,0 -20120,0 -20121,0 -20122,0 -20123,0 -20124,0 -20125,0 -20126,0 -20127,0 -20128,0 -20129,0 -20130,0 -20131,0 -20132,0 -20133,0 -20134,0 -20135,0 -20136,0 -20137,0 -20138,0 -20139,0 -20140,0 -20141,0 -20142,0 -20143,0 -20144,0 -20145,0 -20146,0 -20147,0 -20148,0 -20149,0 -20150,0 -20151,0 -20152,0 -20153,0 -20154,0 -20155,0 -20156,0 -20157,0 -20158,0 -20159,0 -20160,0 -20161,0 -20162,0 -20163,0 -20164,0 -20165,0 -20166,0 -20167,0 -20168,0 -20169,0 -20170,0 -20171,0 -20172,0 -20173,0 -20174,0 -20175,0 -20176,0 -20177,0 -20178,0 -20179,0 -20180,0 -20181,0 -20182,0 -20183,0 -20184,0 -20185,0 -20186,0 -20187,0 -20188,0 -20189,0 -20190,0 -20191,0 -20192,0 -20193,0 -20194,0 -20195,0 -20196,0 -20197,0 -20198,0 -20199,0 -20200,0 -20201,0 -20202,0 -20203,0 -20204,0 -20205,0 -20206,0 -20207,0 -20208,0 -20209,0 -20210,0 -20211,0 -20212,0 -20213,0 -20214,0 -20215,0 -20216,0 -20217,0 -20218,0 -20219,0 -20220,0 -20221,0 -20222,0 -20223,0 -20224,0 -20225,0 -20226,0 -20227,0 -20228,0 -20229,0 -20230,0 -20231,0 -20232,0 -20233,0 -20234,0 -20235,0 -20236,0 -20237,0 -20238,0 -20239,0 -20240,0 -20241,0 -20242,0 -20243,0 -20244,0 -20245,0 -20246,0 -20247,0 -20248,0 -20249,0 -20250,0 -20251,0 -20252,0 -20253,0 -20254,0 -20255,0 -20256,0 -20257,0 -20258,0 -20259,0 -20260,0 -20261,0 -20262,0 -20263,0 -20264,0 -20265,0 -20266,0 -20267,0 -20268,0 -20269,0 -20270,0 -20271,0 -20272,0 -20273,0 -20274,0 -20275,0 -20276,0 -20277,0 -20278,0 -20279,0 -20280,0 -20281,0 -20282,0 -20283,0 -20284,0 -20285,0 -20286,0 -20287,0 -20288,0 -20289,0 -20290,0 -20291,0 -20292,0 -20293,0 -20294,0 -20295,0 -20296,0 -20297,0 -20298,0 -20299,0 -20300,0 -20301,0 -20302,0 -20303,0 -20304,0 -20305,0 -20306,0 -20307,0 -20308,0 -20309,0 -20310,0 -20311,0 -20312,0 -20313,0 -20314,0 -20315,0 -20316,0 -20317,0 -20318,0 -20319,0 -20320,0 -20321,0 -20322,0 -20323,0 -20324,0 -20325,0 -20326,0 -20327,0 -20328,0 -20329,0 -20330,0 -20331,0 -20332,0 -20333,0 -20334,0 -20335,0 -20336,0 -20337,0 -20338,0 -20339,0 -20340,0 -20341,0 -20342,0 -20343,0 -20344,0 -20345,0 -20346,0 -20347,0 -20348,0 -20349,0 -20350,0 -20351,0 -20352,0 -20353,0 -20354,0 -20355,0 -20356,0 -20357,0 -20358,0 -20359,0 -20360,0 -20361,0 -20362,0 -20363,0 -20364,0 -20365,0 -20366,0 -20367,0 -20368,0 -20369,0 -20370,0 -20371,0 -20372,0 -20373,0 -20374,0 -20375,0 -20376,0 -20377,0 -20378,0 -20379,0 -20380,0 -20381,0 -20382,0 -20383,0 -20384,0 -20385,0 -20386,0 -20387,0 -20388,0 -20389,0 -20390,0 -20391,0 -20392,0 -20393,0 -20394,0 -20395,0 -20396,0 -20397,0 -20398,0 -20399,0 -20400,0 -20401,0 -20402,0 -20403,0 -20404,0 -20405,0 -20406,0 -20407,0 -20408,0 -20409,0 -20410,0 -20411,0 -20412,0 -20413,0 -20414,0 -20415,0 -20416,0 -20417,0 -20418,0 -20419,0 -20420,0 -20421,0 -20422,0 -20423,0 -20424,0 -20425,0 -20426,0 -20427,0 -20428,0 -20429,0 -20430,0 -20431,0 -20432,0 -20433,0 -20434,0 -20435,0 -20436,0 -20437,0 -20438,0 -20439,0 -20440,0 -20441,0 -20442,0 -20443,0 -20444,0 -20445,0 -20446,0 -20447,0 -20448,0 -20449,0 -20450,0 -20451,0 -20452,0 -20453,0 -20454,0 -20455,0 -20456,0 -20457,0 -20458,0 -20459,0 -20460,0 -20461,0 -20462,0 -20463,0 -20464,0 -20465,0 -20466,0 -20467,0 -20468,0 -20469,0 -20470,0 -20471,0 -20472,0 -20473,0 -20474,0 -20475,0 -20476,0 -20477,0 -20478,0 -20479,0 -20480,0 -20481,0 -20482,0 -20483,0 -20484,0 -20485,0 -20486,0 -20487,0 -20488,0 -20489,0 -20490,0 -20491,0 -20492,0 -20493,0 -20494,0 -20495,0 -20496,0 -20497,0 -20498,0 -20499,0 -20500,0 -20501,0 -20502,0 -20503,0 -20504,0 -20505,0 -20506,0 -20507,0 -20508,0 -20509,0 -20510,0 -20511,0 -20512,0 -20513,0 -20514,0 -20515,0 -20516,0 -20517,0 -20518,0 -20519,0 -20520,0 -20521,0 -20522,0 -20523,0 -20524,0 -20525,0 -20526,0 -20527,0 -20528,0 -20529,0 -20530,0 -20531,0 -20532,0 -20533,0 -20534,0 -20535,0 -20536,0 -20537,0 -20538,0 -20539,0 -20540,0 -20541,0 -20542,0 -20543,0 -20544,0 -20545,0 -20546,0 -20547,0 -20548,0 -20549,0 -20550,0 -20551,0 -20552,0 -20553,0 -20554,0 -20555,0 -20556,0 -20557,0 -20558,0 -20559,0 -20560,0 -20561,0 -20562,0 -20563,0 -20564,0 -20565,0 -20566,0 -20567,0 -20568,0 -20569,0 -20570,0 -20571,0 -20572,0 -20573,0 -20574,0 -20575,0 -20576,0 -20577,0 -20578,0 -20579,0 -20580,0 -20581,0 -20582,0 -20583,0 -20584,0 -20585,0 -20586,0 -20587,0 -20588,0 -20589,0 -20590,0 -20591,0 -20592,0 -20593,0 -20594,0 -20595,0 -20596,0 -20597,0 -20598,0 -20599,0 -20600,0 -20601,0 -20602,0 -20603,0 -20604,0 -20605,0 -20606,0 -20607,0 -20608,0 -20609,0 -20610,0 -20611,0 -20612,0 -20613,0 -20614,0 -20615,0 -20616,0 -20617,0 -20618,0 -20619,0 -20620,0 -20621,0 -20622,0 -20623,0 -20624,0 -20625,0 -20626,0 -20627,0 -20628,0 -20629,0 -20630,0 -20631,0 -20632,0 -20633,0 -20634,0 -20635,0 -20636,0 -20637,0 -20638,0 -20639,0 -20640,0 -20641,0 -20642,0 -20643,0 -20644,0 -20645,0 -20646,0 -20647,0 -20648,0 -20649,0 -20650,0 -20651,0 -20652,0 -20653,0 -20654,0 -20655,0 -20656,0 -20657,0 -20658,0 -20659,0 -20660,0 -20661,0 -20662,0 -20663,0 -20664,0 -20665,0 -20666,0 -20667,0 -20668,0 -20669,0 -20670,0 -20671,0 -20672,0 -20673,0 -20674,0 -20675,0 -20676,0 -20677,0 -20678,0 -20679,0 -20680,0 -20681,0 -20682,0 -20683,0 -20684,0 -20685,0 -20686,0 -20687,0 -20688,0 -20689,0 -20690,0 -20691,0 -20692,0 -20693,0 -20694,0 -20695,0 -20696,0 -20697,0 -20698,0 -20699,0 -20700,0 -20701,0 -20702,0 -20703,0 -20704,0 -20705,0 -20706,0 -20707,0 -20708,0 -20709,0 -20710,0 -20711,0 -20712,0 -20713,0 -20714,0 -20715,0 -20716,0 -20717,0 -20718,0 -20719,0 -20720,0 -20721,0 -20722,0 -20723,0 -20724,0 -20725,0 -20726,0 -20727,0 -20728,0 -20729,0 -20730,0 -20731,0 -20732,0 -20733,0 -20734,0 -20735,0 -20736,0 -20737,0 -20738,0 -20739,0 -20740,0 -20741,0 -20742,0 -20743,0 -20744,0 -20745,0 -20746,0 -20747,0 -20748,0 -20749,0 -20750,0 -20751,0 -20752,0 -20753,0 -20754,0 -20755,0 -20756,0 -20757,0 -20758,0 -20759,0 -20760,0 -20761,0 -20762,0 -20763,0 -20764,0 -20765,0 -20766,0 -20767,0 -20768,0 -20769,0 -20770,0 -20771,0 -20772,0 -20773,0 -20774,0 -20775,0 -20776,0 -20777,0 -20778,0 -20779,0 -20780,0 -20781,0 -20782,0 -20783,0 -20784,0 -20785,0 -20786,0 -20787,0 -20788,0 -20789,0 -20790,0 -20791,0 -20792,0 -20793,0 -20794,0 -20795,0 -20796,0 -20797,0 -20798,0 -20799,0 -20800,0 -20801,0 -20802,0 -20803,0 -20804,0 -20805,0 -20806,0 -20807,0 -20808,0 -20809,0 -20810,0 -20811,0 -20812,0 -20813,0 -20814,0 -20815,0 -20816,0 -20817,0 -20818,0 -20819,0 -20820,0 -20821,0 -20822,0 -20823,0 -20824,0 -20825,0 -20826,0 -20827,0 -20828,0 -20829,0 -20830,0 -20831,0 -20832,0 -20833,0 -20834,0 -20835,0 -20836,0 -20837,0 -20838,0 -20839,0 -20840,0 -20841,0 -20842,0 -20843,0 -20844,0 -20845,0 -20846,0 -20847,0 -20848,0 -20849,0 -20850,0 -20851,0 -20852,0 -20853,0 -20854,0 -20855,0 -20856,0 -20857,0 -20858,0 -20859,0 -20860,0 -20861,0 -20862,0 -20863,0 -20864,0 -20865,0 -20866,0 -20867,0 -20868,0 -20869,0 -20870,0 -20871,0 -20872,0 -20873,0 -20874,0 -20875,0 -20876,0 -20877,0 -20878,0 -20879,0 -20880,0 -20881,0 -20882,0 -20883,0 -20884,0 -20885,0 -20886,0 -20887,0 -20888,0 -20889,0 -20890,0 -20891,0 -20892,0 -20893,0 -20894,0 -20895,0 -20896,0 -20897,0 -20898,0 -20899,0 -20900,0 -20901,0 -20902,0 -20903,0 -20904,0 -20905,0 -20906,0 -20907,0 -20908,0 -20909,0 -20910,0 -20911,0 -20912,0 -20913,0 -20914,0 -20915,0 -20916,0 -20917,0 -20918,0 -20919,0 -20920,0 -20921,0 -20922,0 -20923,0 -20924,0 -20925,0 -20926,0 -20927,0 -20928,0 -20929,0 -20930,0 -20931,0 -20932,0 -20933,0 -20934,0 -20935,0 -20936,0 -20937,0 -20938,0 -20939,0 -20940,0 -20941,0 -20942,0 -20943,0 -20944,0 -20945,0 -20946,0 -20947,0 -20948,0 -20949,0 -20950,0 -20951,0 -20952,0 -20953,0 -20954,0 -20955,0 -20956,0 -20957,0 -20958,0 -20959,0 -20960,0 -20961,0 -20962,0 -20963,0 -20964,0 -20965,0 -20966,0 -20967,0 -20968,0 -20969,0 -20970,0 -20971,0 -20972,0 -20973,0 -20974,0 -20975,0 -20976,0 -20977,0 -20978,0 -20979,0 -20980,0 -20981,0 -20982,0 -20983,0 -20984,0 -20985,0 -20986,0 -20987,0 -20988,0 -20989,0 -20990,0 -20991,0 -20992,0 -20993,0 -20994,0 -20995,0 -20996,0 -20997,0 -20998,0 -20999,0 -21000,0 -21001,0 -21002,0 -21003,0 -21004,0 -21005,0 -21006,0 -21007,0 -21008,0 -21009,0 -21010,0 -21011,0 -21012,0 -21013,0 -21014,0 -21015,0 -21016,0 -21017,0 -21018,0 -21019,0 -21020,0 -21021,0 -21022,0 -21023,0 -21024,0 -21025,0 -21026,0 -21027,0 -21028,0 -21029,0 -21030,0 -21031,0 -21032,0 -21033,0 -21034,0 -21035,0 -21036,0 -21037,0 -21038,0 -21039,0 -21040,0 -21041,0 -21042,0 -21043,0 -21044,0 -21045,0 -21046,0 -21047,0 -21048,0 -21049,0 -21050,0 -21051,0 -21052,0 -21053,0 -21054,0 -21055,0 -21056,0 -21057,0 -21058,0 -21059,0 -21060,0 -21061,0 -21062,0 -21063,0 -21064,0 -21065,0 -21066,0 -21067,0 -21068,0 -21069,0 -21070,0 -21071,0 -21072,0 -21073,0 -21074,0 -21075,0 -21076,0 -21077,0 -21078,0 -21079,0 -21080,0 -21081,0 -21082,0 -21083,0 -21084,0 -21085,0 -21086,0 -21087,0 -21088,0 -21089,0 -21090,0 -21091,0 -21092,0 -21093,0 -21094,0 -21095,0 -21096,0 -21097,0 -21098,0 -21099,0 -21100,0 -21101,0 -21102,0 -21103,0 -21104,0 -21105,0 -21106,0 -21107,0 -21108,0 -21109,0 -21110,0 -21111,0 -21112,0 -21113,0 -21114,0 -21115,0 -21116,0 -21117,0 -21118,0 -21119,0 -21120,0 -21121,0 -21122,0 -21123,0 -21124,0 -21125,0 -21126,0 -21127,0 -21128,0 -21129,0 -21130,0 -21131,0 -21132,0 -21133,0 -21134,0 -21135,0 -21136,0 -21137,0 -21138,0 -21139,0 -21140,0 -21141,0 -21142,0 -21143,0 -21144,0 -21145,0 -21146,0 -21147,0 -21148,0 -21149,0 -21150,0 -21151,0 -21152,0 -21153,0 -21154,0 -21155,0 -21156,0 -21157,0 -21158,0 -21159,0 -21160,0 -21161,0 -21162,0 -21163,0 -21164,0 -21165,0 -21166,0 -21167,0 -21168,0 -21169,0 -21170,0 -21171,0 -21172,0 -21173,0 -21174,0 -21175,0 -21176,0 -21177,0 -21178,0 -21179,0 -21180,0 -21181,0 -21182,0 -21183,0 -21184,0 -21185,0 -21186,0 -21187,0 -21188,0 -21189,0 -21190,0 -21191,0 -21192,0 -21193,0 -21194,0 -21195,0 -21196,0 -21197,0 -21198,0 -21199,0 -21200,0 -21201,0 -21202,0 -21203,0 -21204,0 -21205,0 -21206,0 -21207,0 -21208,0 -21209,0 -21210,0 -21211,0 -21212,0 -21213,0 -21214,0 -21215,0 -21216,0 -21217,0 -21218,0 -21219,0 -21220,0 -21221,0 -21222,0 -21223,0 -21224,0 -21225,0 -21226,0 -21227,0 -21228,0 -21229,0 -21230,0 -21231,0 -21232,0 -21233,0 -21234,0 -21235,0 -21236,0 -21237,0 -21238,0 -21239,0 -21240,0 -21241,0 -21242,0 -21243,0 -21244,0 -21245,0 -21246,0 -21247,0 -21248,0 -21249,0 -21250,0 -21251,0 -21252,0 -21253,0 -21254,0 -21255,0 -21256,0 -21257,0 -21258,0 -21259,0 -21260,0 -21261,0 -21262,0 -21263,0 -21264,0 -21265,0 -21266,0 -21267,0 -21268,0 -21269,0 -21270,0 -21271,0 -21272,0 -21273,0 -21274,0 -21275,0 -21276,0 -21277,0 -21278,0 -21279,0 -21280,0 -21281,0 -21282,0 -21283,0 -21284,0 -21285,0 -21286,0 -21287,0 -21288,0 -21289,0 -21290,0 -21291,0 -21292,0 -21293,0 -21294,0 -21295,0 -21296,0 -21297,0 -21298,0 -21299,0 -21300,0 -21301,0 -21302,0 -21303,0 -21304,0 -21305,0 -21306,0 -21307,0 -21308,0 -21309,0 -21310,0 -21311,0 -21312,0 -21313,0 -21314,0 -21315,0 -21316,0 -21317,0 -21318,0 -21319,0 -21320,0 -21321,0 -21322,0 -21323,0 -21324,0 -21325,0 -21326,0 -21327,0 -21328,0 -21329,0 -21330,0 -21331,0 -21332,0 -21333,0 -21334,0 -21335,0 -21336,0 -21337,0 -21338,0 -21339,0 -21340,0 -21341,0 -21342,0 -21343,0 -21344,0 -21345,0 -21346,0 -21347,0 -21348,0 -21349,0 -21350,0 -21351,0 -21352,0 -21353,0 -21354,0 -21355,0 -21356,0 -21357,0 -21358,0 -21359,0 -21360,0 -21361,0 -21362,0 -21363,0 -21364,0 -21365,0 -21366,0 -21367,0 -21368,0 -21369,0 -21370,0 -21371,0 -21372,0 -21373,0 -21374,0 -21375,0 -21376,0 -21377,0 -21378,0 -21379,0 -21380,0 -21381,0 -21382,0 -21383,0 -21384,0 -21385,0 -21386,0 -21387,0 -21388,0 -21389,0 -21390,0 -21391,0 -21392,0 -21393,0 -21394,0 -21395,0 -21396,0 -21397,0 -21398,0 -21399,0 -21400,0 -21401,0 -21402,0 -21403,0 -21404,0 -21405,0 -21406,0 -21407,0 -21408,0 -21409,0 -21410,0 -21411,0 -21412,0 -21413,0 -21414,0 -21415,0 -21416,0 -21417,0 -21418,0 -21419,0 -21420,0 -21421,0 -21422,0 -21423,0 -21424,0 -21425,0 -21426,0 -21427,0 -21428,0 -21429,0 -21430,0 -21431,0 -21432,0 -21433,0 -21434,0 -21435,0 -21436,0 -21437,0 -21438,0 -21439,0 -21440,0 -21441,0 -21442,0 -21443,0 -21444,0 -21445,0 -21446,0 -21447,0 -21448,0 -21449,0 -21450,0 -21451,0 -21452,0 -21453,0 -21454,0 -21455,0 -21456,0 -21457,0 -21458,0 -21459,0 -21460,0 -21461,0 -21462,0 -21463,0 -21464,0 -21465,0 -21466,0 -21467,0 -21468,0 -21469,0 -21470,0 -21471,0 -21472,0 -21473,0 -21474,0 -21475,0 -21476,0 -21477,0 -21478,0 -21479,0 -21480,0 -21481,0 -21482,0 -21483,0 -21484,0 -21485,0 -21486,0 -21487,0 -21488,0 -21489,0 -21490,0 -21491,0 -21492,0 -21493,0 -21494,0 -21495,0 -21496,0 -21497,0 -21498,0 -21499,0 -21500,0 -21501,0 -21502,0 -21503,0 -21504,0 -21505,0 -21506,0 -21507,0 -21508,0 -21509,0 -21510,0 -21511,0 -21512,0 -21513,0 -21514,0 -21515,0 -21516,0 -21517,0 -21518,0 -21519,0 -21520,0 -21521,0 -21522,0 -21523,0 -21524,0 -21525,0 -21526,0 -21527,0 -21528,0 -21529,0 -21530,0 -21531,0 -21532,0 -21533,0 -21534,0 -21535,0 -21536,0 -21537,0 -21538,0 -21539,0 -21540,0 -21541,0 -21542,0 -21543,0 -21544,0 -21545,0 -21546,0 -21547,0 -21548,0 -21549,0 -21550,0 -21551,0 -21552,0 -21553,0 -21554,0 -21555,0 -21556,0 -21557,0 -21558,0 -21559,0 -21560,0 -21561,0 -21562,0 -21563,0 -21564,0 -21565,0 -21566,0 -21567,0 -21568,0 -21569,0 -21570,0 -21571,0 -21572,0 -21573,0 -21574,0 -21575,0 -21576,0 -21577,0 -21578,0 -21579,0 -21580,0 -21581,0 -21582,0 -21583,0 -21584,0 -21585,0 -21586,0 -21587,0 -21588,0 -21589,0 -21590,0 -21591,0 -21592,0 -21593,0 -21594,0 -21595,0 -21596,0 -21597,0 -21598,0 -21599,0 -21600,0 -21601,0 -21602,0 -21603,0 -21604,0 -21605,0 -21606,0 -21607,0 -21608,0 -21609,0 -21610,0 -21611,0 -21612,0 -21613,0 -21614,0 -21615,0 -21616,0 -21617,0 -21618,0 -21619,0 -21620,0 -21621,0 -21622,0 -21623,0 -21624,0 -21625,0 -21626,0 -21627,0 -21628,0 -21629,0 -21630,0 -21631,0 -21632,0 -21633,0 -21634,0 -21635,0 -21636,0 -21637,0 -21638,0 -21639,0 -21640,0 -21641,0 -21642,0 -21643,0 -21644,0 -21645,0 -21646,0 -21647,0 -21648,0 -21649,0 -21650,0 -21651,0 -21652,0 -21653,0 -21654,0 -21655,0 -21656,0 -21657,0 -21658,0 -21659,0 -21660,0 -21661,0 -21662,0 -21663,0 -21664,0 -21665,0 -21666,0 -21667,0 -21668,0 -21669,0 -21670,0 -21671,0 -21672,0 -21673,0 -21674,0 -21675,0 -21676,0 -21677,0 -21678,0 -21679,0 -21680,0 -21681,0 -21682,0 -21683,0 -21684,0 -21685,0 -21686,0 -21687,0 -21688,0 -21689,0 -21690,0 -21691,0 -21692,0 -21693,0 -21694,0 -21695,0 -21696,0 -21697,0 -21698,0 -21699,0 -21700,0 -21701,0 -21702,0 -21703,0 -21704,0 -21705,0 -21706,0 -21707,0 -21708,0 -21709,0 -21710,0 -21711,0 -21712,0 -21713,0 -21714,0 -21715,0 -21716,0 -21717,0 -21718,0 -21719,0 -21720,0 -21721,0 -21722,0 -21723,0 -21724,0 -21725,0 -21726,0 -21727,0 -21728,0 -21729,0 -21730,0 -21731,0 -21732,0 -21733,0 -21734,0 -21735,0 -21736,0 -21737,0 -21738,0 -21739,0 -21740,0 -21741,0 -21742,0 -21743,0 -21744,0 -21745,0 -21746,0 -21747,0 -21748,0 -21749,0 -21750,0 -21751,0 -21752,0 -21753,0 -21754,0 -21755,0 -21756,0 -21757,0 -21758,0 -21759,0 -21760,0 -21761,0 -21762,0 -21763,0 -21764,0 -21765,0 -21766,0 -21767,0 -21768,0 -21769,0 -21770,0 -21771,0 -21772,0 -21773,0 -21774,0 -21775,0 -21776,0 -21777,0 -21778,0 -21779,0 -21780,0 -21781,0 -21782,0 -21783,0 -21784,0 -21785,0 -21786,0 -21787,0 -21788,0 -21789,0 -21790,0 -21791,0 -21792,0 -21793,0 -21794,0 -21795,0 -21796,0 -21797,0 -21798,0 -21799,0 -21800,0 -21801,0 -21802,0 -21803,0 -21804,0 -21805,0 -21806,0 -21807,0 -21808,0 -21809,0 -21810,0 -21811,0 -21812,0 -21813,0 -21814,0 -21815,0 -21816,0 -21817,0 -21818,0 -21819,0 -21820,0 -21821,0 -21822,0 -21823,0 -21824,0 -21825,0 -21826,0 -21827,0 -21828,0 -21829,0 -21830,0 -21831,0 -21832,0 -21833,0 -21834,0 -21835,0 -21836,0 -21837,0 -21838,0 -21839,0 -21840,0 -21841,0 -21842,0 -21843,0 -21844,0 -21845,0 -21846,0 -21847,0 -21848,0 -21849,0 -21850,0 -21851,0 -21852,0 -21853,0 -21854,0 -21855,0 -21856,0 -21857,0 -21858,0 -21859,0 -21860,0 -21861,0 -21862,0 -21863,0 -21864,0 -21865,0 -21866,0 -21867,0 -21868,0 -21869,0 -21870,0 -21871,0 -21872,0 -21873,0 -21874,0 -21875,0 -21876,0 -21877,0 -21878,0 -21879,0 -21880,0 -21881,0 -21882,0 -21883,0 -21884,0 -21885,0 -21886,0 -21887,0 -21888,0 -21889,0 -21890,0 -21891,0 -21892,0 -21893,0 -21894,0 -21895,0 -21896,0 -21897,0 -21898,0 -21899,0 -21900,0 -21901,0 -21902,0 -21903,0 -21904,0 -21905,0 -21906,0 -21907,0 -21908,0 -21909,0 -21910,0 -21911,0 -21912,0 -21913,0 -21914,0 -21915,0 -21916,0 -21917,0 -21918,0 -21919,0 -21920,0 -21921,0 -21922,0 -21923,0 -21924,0 -21925,0 -21926,0 -21927,0 -21928,0 -21929,0 -21930,0 -21931,0 -21932,0 -21933,0 -21934,0 -21935,0 -21936,0 -21937,0 -21938,0 -21939,0 -21940,0 -21941,0 -21942,0 -21943,0 -21944,0 -21945,0 -21946,0 -21947,0 -21948,0 -21949,0 -21950,0 -21951,0 -21952,0 -21953,0 -21954,0 -21955,0 -21956,0 -21957,0 -21958,0 -21959,0 -21960,0 -21961,0 -21962,0 -21963,0 -21964,0 -21965,0 -21966,0 -21967,0 -21968,0 -21969,0 -21970,0 -21971,0 -21972,0 -21973,0 -21974,0 -21975,0 -21976,0 -21977,0 -21978,0 -21979,0 -21980,0 -21981,0 -21982,0 -21983,0 -21984,0 -21985,0 -21986,0 -21987,0 -21988,0 -21989,0 -21990,0 -21991,0 -21992,0 -21993,0 -21994,0 -21995,0 -21996,0 -21997,0 -21998,0 -21999,0 -22000,0 -22001,0 -22002,0 -22003,0 -22004,0 -22005,0 -22006,0 -22007,0 -22008,0 -22009,0 -22010,0 -22011,0 -22012,0 -22013,0 -22014,0 -22015,0 -22016,0 -22017,0 -22018,0 -22019,0 -22020,0 -22021,0 -22022,0 -22023,0 -22024,0 -22025,0 -22026,0 -22027,0 -22028,0 -22029,0 -22030,0 -22031,0 -22032,0 -22033,0 -22034,0 -22035,0 -22036,0 -22037,0 -22038,0 -22039,0 -22040,0 -22041,0 -22042,0 -22043,0 -22044,0 -22045,0 -22046,0 -22047,0 -22048,0 -22049,0 -22050,0 -22051,0 -22052,0 -22053,0 -22054,0 -22055,0 -22056,0 -22057,0 -22058,0 -22059,0 -22060,0 -22061,0 -22062,0 -22063,0 -22064,0 -22065,0 -22066,0 -22067,0 -22068,0 -22069,0 -22070,0 -22071,0 -22072,0 -22073,0 -22074,0 -22075,0 -22076,0 -22077,0 -22078,0 -22079,0 -22080,0 -22081,0 -22082,0 -22083,0 -22084,0 -22085,0 -22086,0 -22087,0 -22088,0 -22089,0 -22090,0 -22091,0 -22092,0 -22093,0 -22094,0 -22095,0 -22096,0 -22097,0 -22098,0 -22099,0 -22100,0 -22101,0 -22102,0 -22103,0 -22104,0 -22105,0 -22106,0 -22107,0 -22108,0 -22109,0 -22110,0 -22111,0 -22112,0 -22113,0 -22114,0 -22115,0 -22116,0 -22117,0 -22118,0 -22119,0 -22120,0 -22121,0 -22122,0 -22123,0 -22124,0 -22125,0 -22126,0 -22127,0 -22128,0 -22129,0 -22130,0 -22131,0 -22132,0 -22133,0 -22134,0 -22135,0 -22136,0 -22137,0 -22138,0 -22139,0 -22140,0 -22141,0 -22142,0 -22143,0 -22144,0 -22145,0 -22146,0 -22147,0 -22148,0 -22149,0 -22150,0 -22151,0 -22152,0 -22153,0 -22154,0 -22155,0 -22156,0 -22157,0 -22158,0 -22159,0 -22160,0 -22161,0 -22162,0 -22163,0 -22164,0 -22165,0 -22166,0 -22167,0 -22168,0 -22169,0 -22170,0 -22171,0 -22172,0 -22173,0 -22174,0 -22175,0 -22176,0 -22177,0 -22178,0 -22179,0 -22180,0 -22181,0 -22182,0 -22183,0 -22184,0 -22185,0 -22186,0 -22187,0 -22188,0 -22189,0 -22190,0 -22191,0 -22192,0 -22193,0 -22194,0 -22195,0 -22196,0 -22197,0 -22198,0 -22199,0 -22200,0 -22201,0 -22202,0 -22203,0 -22204,0 -22205,0 -22206,0 -22207,0 -22208,0 -22209,0 -22210,0 -22211,0 -22212,0 -22213,0 -22214,0 -22215,0 -22216,0 -22217,0 -22218,0 -22219,0 -22220,0 -22221,0 -22222,0 -22223,0 -22224,0 -22225,0 -22226,0 -22227,0 -22228,0 -22229,0 -22230,0 -22231,0 -22232,0 -22233,0 -22234,0 -22235,0 -22236,0 -22237,0 -22238,0 -22239,0 -22240,0 -22241,0 -22242,0 -22243,0 -22244,0 -22245,0 -22246,0 -22247,0 -22248,0 -22249,0 -22250,0 -22251,0 -22252,0 -22253,0 -22254,0 -22255,0 -22256,0 -22257,0 -22258,0 -22259,0 -22260,0 -22261,0 -22262,0 -22263,0 -22264,0 -22265,0 -22266,0 -22267,0 -22268,0 -22269,0 -22270,0 -22271,0 -22272,0 -22273,0 -22274,0 -22275,0 -22276,0 -22277,0 -22278,0 -22279,0 -22280,0 -22281,0 -22282,0 -22283,0 -22284,0 -22285,0 -22286,0 -22287,0 -22288,0 -22289,0 -22290,0 -22291,0 -22292,0 -22293,0 -22294,0 -22295,0 -22296,0 -22297,0 -22298,0 -22299,0 -22300,0 -22301,0 -22302,0 -22303,0 -22304,0 -22305,0 -22306,0 -22307,0 -22308,0 -22309,0 -22310,0 -22311,0 -22312,0 -22313,0 -22314,0 -22315,0 -22316,0 -22317,0 -22318,0 -22319,0 -22320,0 -22321,0 -22322,0 -22323,0 -22324,0 -22325,0 -22326,0 -22327,0 -22328,0 -22329,0 -22330,0 -22331,0 -22332,0 -22333,0 -22334,0 -22335,0 -22336,0 -22337,0 -22338,0 -22339,0 -22340,0 -22341,0 -22342,0 -22343,0 -22344,0 -22345,0 -22346,0 -22347,0 -22348,0 -22349,0 -22350,0 -22351,0 -22352,0 -22353,0 -22354,0 -22355,0 -22356,0 -22357,0 -22358,0 -22359,0 -22360,0 -22361,0 -22362,0 -22363,0 -22364,0 -22365,0 -22366,0 -22367,0 -22368,0 -22369,0 -22370,0 -22371,0 -22372,0 -22373,0 -22374,0 -22375,0 -22376,0 -22377,0 -22378,0 -22379,0 -22380,0 -22381,0 -22382,0 -22383,0 -22384,0 -22385,0 -22386,0 -22387,0 -22388,0 -22389,0 -22390,0 -22391,0 -22392,0 -22393,0 -22394,0 -22395,0 -22396,0 -22397,0 -22398,0 -22399,0 -22400,0 -22401,0 -22402,0 -22403,0 -22404,0 -22405,0 -22406,0 -22407,0 -22408,0 -22409,0 -22410,0 -22411,0 -22412,0 -22413,0 -22414,0 -22415,0 -22416,0 -22417,0 -22418,0 -22419,0 -22420,0 -22421,0 -22422,0 -22423,0 -22424,0 -22425,0 -22426,0 -22427,0 -22428,0 -22429,0 -22430,0 -22431,0 -22432,0 -22433,0 -22434,0 -22435,0 -22436,0 -22437,0 -22438,0 -22439,0 -22440,0 -22441,0 -22442,0 -22443,0 -22444,0 -22445,0 -22446,0 -22447,0 -22448,0 -22449,0 -22450,0 -22451,0 -22452,0 -22453,0 -22454,0 -22455,0 -22456,0 -22457,0 -22458,0 -22459,0 -22460,0 -22461,0 -22462,0 -22463,0 -22464,0 -22465,0 -22466,0 -22467,0 -22468,0 -22469,0 -22470,0 -22471,0 -22472,0 -22473,0 -22474,0 -22475,0 -22476,0 -22477,0 -22478,0 -22479,0 -22480,0 -22481,0 -22482,0 -22483,0 -22484,0 -22485,0 -22486,0 -22487,0 -22488,0 -22489,0 -22490,0 -22491,0 -22492,0 -22493,0 -22494,0 -22495,0 -22496,0 -22497,0 -22498,0 -22499,0 -22500,0 -22501,0 -22502,0 -22503,0 -22504,0 -22505,0 -22506,0 -22507,0 -22508,0 -22509,0 -22510,0 -22511,0 -22512,0 -22513,0 -22514,0 -22515,0 -22516,0 -22517,0 -22518,0 -22519,0 -22520,0 -22521,0 -22522,0 -22523,0 -22524,0 -22525,0 -22526,0 -22527,0 -22528,0 -22529,0 -22530,0 -22531,0 -22532,0 -22533,0 -22534,0 -22535,0 -22536,0 -22537,0 -22538,0 -22539,0 -22540,0 -22541,0 -22542,0 -22543,0 -22544,0 -22545,0 -22546,0 -22547,0 -22548,0 -22549,0 -22550,0 -22551,0 -22552,0 -22553,0 -22554,0 -22555,0 -22556,0 -22557,0 -22558,0 -22559,0 -22560,0 -22561,0 -22562,0 -22563,0 -22564,0 -22565,0 -22566,0 -22567,0 -22568,0 -22569,0 -22570,0 -22571,0 -22572,0 -22573,0 -22574,0 -22575,0 -22576,0 -22577,0 -22578,0 -22579,0 -22580,0 -22581,0 -22582,0 -22583,0 -22584,0 -22585,0 -22586,0 -22587,0 -22588,0 -22589,0 -22590,0 -22591,0 -22592,0 -22593,0 -22594,0 -22595,0 -22596,0 -22597,0 -22598,0 -22599,0 -22600,0 -22601,0 -22602,0 -22603,0 -22604,0 -22605,0 -22606,0 -22607,0 -22608,0 -22609,0 -22610,0 -22611,0 -22612,0 -22613,0 -22614,0 -22615,0 -22616,0 -22617,0 -22618,0 -22619,0 -22620,0 -22621,0 -22622,0 -22623,0 -22624,0 -22625,0 -22626,0 -22627,0 -22628,0 -22629,0 -22630,0 -22631,0 -22632,0 -22633,0 -22634,0 -22635,0 -22636,0 -22637,0 -22638,0 -22639,0 -22640,0 -22641,0 -22642,0 -22643,0 -22644,0 -22645,0 -22646,0 -22647,0 -22648,0 -22649,0 -22650,0 -22651,0 -22652,0 -22653,0 -22654,0 -22655,0 -22656,0 -22657,0 -22658,0 -22659,0 -22660,0 -22661,0 -22662,0 -22663,0 -22664,0 -22665,0 -22666,0 -22667,0 -22668,0 -22669,0 -22670,0 -22671,0 -22672,0 -22673,0 -22674,0 -22675,0 -22676,0 -22677,0 -22678,0 -22679,0 -22680,0 -22681,0 -22682,0 -22683,0 -22684,0 -22685,0 -22686,0 -22687,0 -22688,0 -22689,0 -22690,0 -22691,0 -22692,0 -22693,0 -22694,0 -22695,0 -22696,0 -22697,0 -22698,0 -22699,0 -22700,0 -22701,0 -22702,0 -22703,0 -22704,0 -22705,0 -22706,0 -22707,0 -22708,0 -22709,0 -22710,0 -22711,0 -22712,0 -22713,0 -22714,0 -22715,0 -22716,0 -22717,0 -22718,0 -22719,0 -22720,0 -22721,0 -22722,0 -22723,0 -22724,0 -22725,0 -22726,0 -22727,0 -22728,0 -22729,0 -22730,0 -22731,0 -22732,0 -22733,0 -22734,0 -22735,0 -22736,0 -22737,0 -22738,0 -22739,0 -22740,0 -22741,0 -22742,0 -22743,0 -22744,0 -22745,0 -22746,0 -22747,0 -22748,0 -22749,0 -22750,0 -22751,0 -22752,0 -22753,0 -22754,0 -22755,0 -22756,0 -22757,0 -22758,0 -22759,0 -22760,0 -22761,0 -22762,0 -22763,0 -22764,0 -22765,0 -22766,0 -22767,0 -22768,0 -22769,0 -22770,0 -22771,0 -22772,0 -22773,0 -22774,0 -22775,0 -22776,0 -22777,0 -22778,0 -22779,0 -22780,0 -22781,0 -22782,0 -22783,0 -22784,0 -22785,0 -22786,0 -22787,0 -22788,0 -22789,0 -22790,0 -22791,0 -22792,0 -22793,0 -22794,0 -22795,0 -22796,0 -22797,0 -22798,0 -22799,0 -22800,0 -22801,0 -22802,0 -22803,0 -22804,0 -22805,0 -22806,0 -22807,0 -22808,0 -22809,0 -22810,0 -22811,0 -22812,0 -22813,0 -22814,0 -22815,0 -22816,0 -22817,0 -22818,0 -22819,0 -22820,0 -22821,0 -22822,0 -22823,0 -22824,0 -22825,0 -22826,0 -22827,0 -22828,0 -22829,0 -22830,0 -22831,0 -22832,0 -22833,0 -22834,0 -22835,0 -22836,0 -22837,0 -22838,0 -22839,0 -22840,0 -22841,0 -22842,0 -22843,0 -22844,0 -22845,0 -22846,0 -22847,0 -22848,0 -22849,0 -22850,0 -22851,0 -22852,0 -22853,0 -22854,0 -22855,0 -22856,0 -22857,0 -22858,0 -22859,0 -22860,0 -22861,0 -22862,0 -22863,0 -22864,0 -22865,0 -22866,0 -22867,0 -22868,0 -22869,0 -22870,0 -22871,0 -22872,0 -22873,0 -22874,0 -22875,0 -22876,0 -22877,0 -22878,0 -22879,0 -22880,0 -22881,0 -22882,0 -22883,0 -22884,0 -22885,0 -22886,0 -22887,0 -22888,0 -22889,0 -22890,0 -22891,0 -22892,0 -22893,0 -22894,0 -22895,0 -22896,0 -22897,0 -22898,0 -22899,0 -22900,0 -22901,0 -22902,0 -22903,0 -22904,0 -22905,0 -22906,0 -22907,0 -22908,0 -22909,0 -22910,0 -22911,0 -22912,0 -22913,0 -22914,0 -22915,0 -22916,0 -22917,0 -22918,0 -22919,0 -22920,0 -22921,0 -22922,0 -22923,0 -22924,0 -22925,0 -22926,0 -22927,0 -22928,0 -22929,0 -22930,0 -22931,0 -22932,0 -22933,0 -22934,0 -22935,0 -22936,0 -22937,0 -22938,0 -22939,0 -22940,0 -22941,0 -22942,0 -22943,0 -22944,0 -22945,0 -22946,0 -22947,0 -22948,0 -22949,0 -22950,0 -22951,0 -22952,0 -22953,0 -22954,0 -22955,0 -22956,0 -22957,0 -22958,0 -22959,0 -22960,0 -22961,0 -22962,0 -22963,0 -22964,0 -22965,0 -22966,0 -22967,0 -22968,0 -22969,0 -22970,0 -22971,0 -22972,0 -22973,0 -22974,0 -22975,0 -22976,0 -22977,0 -22978,0 -22979,0 -22980,0 -22981,0 -22982,0 -22983,0 -22984,0 -22985,0 -22986,0 -22987,0 -22988,0 -22989,0 -22990,0 -22991,0 -22992,0 -22993,0 -22994,0 -22995,0 -22996,0 -22997,0 -22998,0 -22999,0 -23000,0 -23001,0 -23002,0 -23003,0 -23004,0 -23005,0 -23006,0 -23007,0 -23008,0 -23009,0 -23010,0 -23011,0 -23012,0 -23013,0 -23014,0 -23015,0 -23016,0 -23017,0 -23018,0 -23019,0 -23020,0 -23021,0 -23022,0 -23023,0 -23024,0 -23025,0 -23026,0 -23027,0 -23028,0 -23029,0 -23030,0 -23031,0 -23032,0 -23033,0 -23034,0 -23035,0 -23036,0 -23037,0 -23038,0 -23039,0 -23040,0 -23041,0 -23042,0 -23043,0 -23044,0 -23045,0 -23046,0 -23047,0 -23048,0 -23049,0 -23050,0 -23051,0 -23052,0 -23053,0 -23054,0 -23055,0 -23056,0 -23057,0 -23058,0 -23059,0 -23060,0 -23061,0 -23062,0 -23063,0 -23064,0 -23065,0 -23066,0 -23067,0 -23068,0 -23069,0 -23070,0 -23071,0 -23072,0 -23073,0 -23074,0 -23075,0 -23076,0 -23077,0 -23078,0 -23079,0 -23080,0 -23081,0 -23082,0 -23083,0 -23084,0 -23085,0 -23086,0 -23087,0 -23088,0 -23089,0 -23090,0 -23091,0 -23092,0 -23093,0 -23094,0 -23095,0 -23096,0 -23097,0 -23098,0 -23099,0 -23100,0 -23101,0 -23102,0 -23103,0 -23104,0 -23105,0 -23106,0 -23107,0 -23108,0 -23109,0 -23110,0 -23111,0 -23112,0 -23113,0 -23114,0 -23115,0 -23116,0 -23117,0 -23118,0 -23119,0 -23120,0 -23121,0 -23122,0 -23123,0 -23124,0 -23125,0 -23126,0 -23127,0 -23128,0 -23129,0 -23130,0 -23131,0 -23132,0 -23133,0 -23134,0 -23135,0 -23136,0 -23137,0 -23138,0 -23139,0 -23140,0 -23141,0 -23142,0 -23143,0 -23144,0 -23145,0 -23146,0 -23147,0 -23148,0 -23149,0 -23150,0 -23151,0 -23152,0 -23153,0 -23154,0 -23155,0 -23156,0 -23157,0 -23158,0 -23159,0 -23160,0 -23161,0 -23162,0 -23163,0 -23164,0 -23165,0 -23166,0 -23167,0 -23168,0 -23169,0 -23170,0 -23171,0 -23172,0 -23173,0 -23174,0 -23175,0 -23176,0 -23177,0 -23178,0 -23179,0 -23180,0 -23181,0 -23182,0 -23183,0 -23184,0 -23185,0 -23186,0 -23187,0 -23188,0 -23189,0 -23190,0 -23191,0 -23192,0 -23193,0 -23194,0 -23195,0 -23196,0 -23197,0 -23198,0 -23199,0 -23200,0 -23201,0 -23202,0 -23203,0 -23204,0 -23205,0 -23206,0 -23207,0 -23208,0 -23209,0 -23210,0 -23211,0 -23212,0 -23213,0 -23214,0 -23215,0 -23216,0 -23217,0 -23218,0 -23219,0 -23220,0 -23221,0 -23222,0 -23223,0 -23224,0 -23225,0 -23226,0 -23227,0 -23228,0 -23229,0 -23230,0 -23231,0 -23232,0 -23233,0 -23234,0 -23235,0 -23236,0 -23237,0 -23238,0 -23239,0 -23240,0 -23241,0 -23242,0 -23243,0 -23244,0 -23245,0 -23246,0 -23247,0 -23248,0 -23249,0 -23250,0 -23251,0 -23252,0 -23253,0 -23254,0 -23255,0 -23256,0 -23257,0 -23258,0 -23259,0 -23260,0 -23261,0 -23262,0 -23263,0 -23264,0 -23265,0 -23266,0 -23267,0 -23268,0 -23269,0 -23270,0 -23271,0 -23272,0 -23273,0 -23274,0 -23275,0 -23276,0 -23277,0 -23278,0 -23279,0 -23280,0 -23281,0 -23282,0 -23283,0 -23284,0 -23285,0 -23286,0 -23287,0 -23288,0 -23289,0 -23290,0 -23291,0 -23292,0 -23293,0 -23294,0 -23295,0 -23296,0 -23297,0 -23298,0 -23299,0 -23300,0 -23301,0 -23302,0 -23303,0 -23304,0 -23305,0 -23306,0 -23307,0 -23308,0 -23309,0 -23310,0 -23311,0 -23312,0 -23313,0 -23314,0 -23315,0 -23316,0 -23317,0 -23318,0 -23319,0 -23320,0 -23321,0 -23322,0 -23323,0 -23324,0 -23325,0 -23326,0 -23327,0 -23328,0 -23329,0 -23330,0 -23331,0 -23332,0 -23333,0 -23334,0 -23335,0 -23336,0 -23337,0 -23338,0 -23339,0 -23340,0 -23341,0 -23342,0 -23343,0 -23344,0 -23345,0 -23346,0 -23347,0 -23348,0 -23349,0 -23350,0 -23351,0 -23352,0 -23353,0 -23354,0 -23355,0 -23356,0 -23357,0 -23358,0 -23359,0 -23360,0 -23361,0 -23362,0 -23363,0 -23364,0 -23365,0 -23366,0 -23367,0 -23368,0 -23369,0 -23370,0 -23371,0 -23372,0 -23373,0 -23374,0 -23375,0 -23376,0 -23377,0 -23378,0 -23379,0 -23380,0 -23381,0 -23382,0 -23383,0 -23384,0 -23385,0 -23386,0 -23387,0 -23388,0 -23389,0 -23390,0 -23391,0 -23392,0 -23393,0 -23394,0 -23395,0 -23396,0 -23397,0 -23398,0 -23399,0 -23400,0 -23401,0 -23402,0 -23403,0 -23404,0 -23405,0 -23406,0 -23407,0 -23408,0 -23409,0 -23410,0 -23411,0 -23412,0 -23413,0 -23414,0 -23415,0 -23416,0 -23417,0 -23418,0 -23419,0 -23420,0 -23421,0 -23422,0 -23423,0 -23424,0 -23425,0 -23426,0 -23427,0 -23428,0 -23429,0 -23430,0 -23431,0 -23432,0 -23433,0 -23434,0 -23435,0 -23436,0 -23437,0 -23438,0 -23439,0 -23440,0 -23441,0 -23442,0 -23443,0 -23444,0 -23445,0 -23446,0 -23447,0 -23448,0 -23449,0 -23450,0 -23451,0 -23452,0 -23453,0 -23454,0 -23455,0 -23456,0 -23457,0 -23458,0 -23459,0 -23460,0 -23461,0 -23462,0 -23463,0 -23464,0 -23465,0 -23466,0 -23467,0 -23468,0 -23469,0 -23470,0 -23471,0 -23472,0 -23473,0 -23474,0 -23475,0 -23476,0 -23477,0 -23478,0 -23479,0 -23480,0 -23481,0 -23482,0 -23483,0 -23484,0 -23485,0 -23486,0 -23487,0 -23488,0 -23489,0 -23490,0 -23491,0 -23492,0 -23493,0 -23494,0 -23495,0 -23496,0 -23497,0 -23498,0 -23499,0 -23500,0 -23501,0 -23502,0 -23503,0 -23504,0 -23505,0 -23506,0 -23507,0 -23508,0 -23509,0 -23510,0 -23511,0 -23512,0 -23513,0 -23514,0 -23515,0 -23516,0 -23517,0 -23518,0 -23519,0 -23520,0 -23521,0 -23522,0 -23523,0 -23524,0 -23525,0 -23526,0 -23527,0 -23528,0 -23529,0 -23530,0 -23531,0 -23532,0 -23533,0 -23534,0 -23535,0 -23536,0 -23537,0 -23538,0 -23539,0 -23540,0 -23541,0 -23542,0 -23543,0 -23544,0 -23545,0 -23546,0 -23547,0 -23548,0 -23549,0 -23550,0 -23551,0 -23552,0 -23553,0 -23554,0 -23555,0 -23556,0 -23557,0 -23558,0 -23559,0 -23560,0 -23561,0 -23562,0 -23563,0 -23564,0 -23565,0 -23566,0 -23567,0 -23568,0 -23569,0 -23570,0 -23571,0 -23572,0 -23573,0 -23574,0 -23575,0 -23576,0 -23577,0 -23578,0 -23579,0 -23580,0 -23581,0 -23582,0 -23583,0 -23584,0 -23585,0 -23586,0 -23587,0 -23588,0 -23589,0 -23590,0 -23591,0 -23592,0 -23593,0 -23594,0 -23595,0 -23596,0 -23597,0 -23598,0 -23599,0 -23600,0 -23601,0 -23602,0 -23603,0 -23604,0 -23605,0 -23606,0 -23607,0 -23608,0 -23609,0 -23610,0 -23611,0 -23612,0 -23613,0 -23614,0 -23615,0 -23616,0 -23617,0 -23618,0 -23619,0 -23620,0 -23621,0 -23622,0 -23623,0 -23624,0 -23625,0 -23626,0 -23627,0 -23628,0 -23629,0 -23630,0 -23631,0 -23632,0 -23633,0 -23634,0 -23635,0 -23636,0 -23637,0 -23638,0 -23639,0 -23640,0 -23641,0 -23642,0 -23643,0 -23644,0 -23645,0 -23646,0 -23647,0 -23648,0 -23649,0 -23650,0 -23651,0 -23652,0 -23653,0 -23654,0 -23655,0 -23656,0 -23657,0 -23658,0 -23659,0 -23660,0 -23661,0 -23662,0 -23663,0 -23664,0 -23665,0 -23666,0 -23667,0 -23668,0 -23669,0 -23670,0 -23671,0 -23672,0 -23673,0 -23674,0 -23675,0 -23676,0 -23677,0 -23678,0 -23679,0 -23680,0 -23681,0 -23682,0 -23683,0 -23684,0 -23685,0 -23686,0 -23687,0 -23688,0 -23689,0 -23690,0 -23691,0 -23692,0 -23693,0 -23694,0 -23695,0 -23696,0 -23697,0 -23698,0 -23699,0 -23700,0 -23701,0 -23702,0 -23703,0 -23704,0 -23705,0 -23706,0 -23707,0 -23708,0 -23709,0 -23710,0 -23711,0 -23712,0 -23713,0 -23714,0 -23715,0 -23716,0 -23717,0 -23718,0 -23719,0 -23720,0 -23721,0 -23722,0 -23723,0 -23724,0 -23725,0 -23726,0 -23727,0 -23728,0 -23729,0 -23730,0 -23731,0 -23732,0 -23733,0 -23734,0 -23735,0 -23736,0 -23737,0 -23738,0 -23739,0 -23740,0 -23741,0 -23742,0 -23743,0 -23744,0 -23745,0 -23746,0 -23747,0 -23748,0 -23749,0 -23750,0 -23751,0 -23752,0 -23753,0 -23754,0 -23755,0 -23756,0 -23757,0 -23758,0 -23759,0 -23760,0 -23761,0 -23762,0 -23763,0 -23764,0 -23765,0 -23766,0 -23767,0 -23768,0 -23769,0 -23770,0 -23771,0 -23772,0 -23773,0 -23774,0 -23775,0 -23776,0 -23777,0 -23778,0 -23779,0 -23780,0 -23781,0 -23782,0 -23783,0 -23784,0 -23785,0 -23786,0 -23787,0 -23788,0 -23789,0 -23790,0 -23791,0 -23792,0 -23793,0 -23794,0 -23795,0 -23796,0 -23797,0 -23798,0 -23799,0 -23800,0 -23801,0 -23802,0 -23803,0 -23804,0 -23805,0 -23806,0 -23807,0 -23808,0 -23809,0 -23810,0 -23811,0 -23812,0 -23813,0 -23814,0 -23815,0 -23816,0 -23817,0 -23818,0 -23819,0 -23820,0 -23821,0 -23822,0 -23823,0 -23824,0 -23825,0 -23826,0 -23827,0 -23828,0 -23829,0 -23830,0 -23831,0 -23832,0 -23833,0 -23834,0 -23835,0 -23836,0 -23837,0 -23838,0 -23839,0 -23840,0 -23841,0 -23842,0 -23843,0 -23844,0 -23845,0 -23846,0 -23847,0 -23848,0 -23849,0 -23850,0 -23851,0 -23852,0 -23853,0 -23854,0 -23855,0 -23856,0 -23857,0 -23858,0 -23859,0 -23860,0 -23861,0 -23862,0 -23863,0 -23864,0 -23865,0 -23866,0 -23867,0 -23868,0 -23869,0 -23870,0 -23871,0 -23872,0 -23873,0 -23874,0 -23875,0 -23876,0 -23877,0 -23878,0 -23879,0 -23880,0 -23881,0 -23882,0 -23883,0 -23884,0 -23885,0 -23886,0 -23887,0 -23888,0 -23889,0 -23890,0 -23891,0 -23892,0 -23893,0 -23894,0 -23895,0 -23896,0 -23897,0 -23898,0 -23899,0 -23900,0 -23901,0 -23902,0 -23903,0 -23904,0 -23905,0 -23906,0 -23907,0 -23908,0 -23909,0 -23910,0 -23911,0 -23912,0 -23913,0 -23914,0 -23915,0 -23916,0 -23917,0 -23918,0 -23919,0 -23920,0 -23921,0 -23922,0 -23923,0 -23924,0 -23925,0 -23926,0 -23927,0 -23928,0 -23929,0 -23930,0 -23931,0 -23932,0 -23933,0 -23934,0 -23935,0 -23936,0 -23937,0 -23938,0 -23939,0 -23940,0 -23941,0 -23942,0 -23943,0 -23944,0 -23945,0 -23946,0 -23947,0 -23948,0 -23949,0 -23950,0 -23951,0 -23952,0 -23953,0 -23954,0 -23955,0 -23956,0 -23957,0 -23958,0 -23959,0 -23960,0 -23961,0 -23962,0 -23963,0 -23964,0 -23965,0 -23966,0 -23967,0 -23968,0 -23969,0 -23970,0 -23971,0 -23972,0 -23973,0 -23974,0 -23975,0 -23976,0 -23977,0 -23978,0 -23979,0 -23980,0 -23981,0 -23982,0 -23983,0 -23984,0 -23985,0 -23986,0 -23987,0 -23988,0 -23989,0 -23990,0 -23991,0 -23992,0 -23993,0 -23994,0 -23995,0 -23996,0 -23997,0 -23998,0 -23999,0 -24000,0 -24001,0 -24002,0 -24003,0 -24004,0 -24005,0 -24006,0 -24007,0 -24008,0 -24009,0 -24010,0 -24011,0 -24012,0 -24013,0 -24014,0 -24015,0 -24016,0 -24017,0 -24018,0 -24019,0 -24020,0 -24021,0 -24022,0 -24023,0 -24024,0 -24025,0 -24026,0 -24027,0 -24028,0 -24029,0 -24030,0 -24031,0 -24032,0 -24033,0 -24034,0 -24035,0 -24036,0 -24037,0 -24038,0 -24039,0 -24040,0 -24041,0 -24042,0 -24043,0 -24044,0 -24045,0 -24046,0 -24047,0 -24048,0 -24049,0 -24050,0 -24051,0 -24052,0 -24053,0 -24054,0 -24055,0 -24056,0 -24057,0 -24058,0 -24059,0 -24060,0 -24061,0 -24062,0 -24063,0 -24064,0 -24065,0 -24066,0 -24067,0 -24068,0 -24069,0 -24070,0 -24071,0 -24072,0 -24073,0 -24074,0 -24075,0 -24076,0 -24077,0 -24078,0 -24079,0 -24080,0 -24081,0 -24082,0 -24083,0 -24084,0 -24085,0 -24086,0 -24087,0 -24088,0 -24089,0 -24090,0 -24091,0 -24092,0 -24093,0 -24094,0 -24095,0 -24096,0 -24097,0 -24098,0 -24099,0 -24100,0 -24101,0 -24102,0 -24103,0 -24104,0 -24105,0 -24106,0 -24107,0 -24108,0 -24109,0 -24110,0 -24111,0 -24112,0 -24113,0 -24114,0 -24115,0 -24116,0 -24117,0 -24118,0 -24119,0 -24120,0 -24121,0 -24122,0 -24123,0 -24124,0 -24125,0 -24126,0 -24127,0 -24128,0 -24129,0 -24130,0 -24131,0 -24132,0 -24133,0 -24134,0 -24135,0 -24136,0 -24137,0 -24138,0 -24139,0 -24140,0 -24141,0 -24142,0 -24143,0 -24144,0 -24145,0 -24146,0 -24147,0 -24148,0 -24149,0 -24150,0 -24151,0 -24152,0 -24153,0 -24154,0 -24155,0 -24156,0 -24157,0 -24158,0 -24159,0 -24160,0 -24161,0 -24162,0 -24163,0 -24164,0 -24165,0 -24166,0 -24167,0 -24168,0 -24169,0 -24170,0 -24171,0 -24172,0 -24173,0 -24174,0 -24175,0 -24176,0 -24177,0 -24178,0 -24179,0 -24180,0 -24181,0 -24182,0 -24183,0 -24184,0 -24185,0 -24186,0 -24187,0 -24188,0 -24189,0 -24190,0 -24191,0 -24192,0 -24193,0 -24194,0 -24195,0 -24196,0 -24197,0 -24198,0 -24199,0 -24200,0 -24201,0 -24202,0 -24203,0 -24204,0 -24205,0 -24206,0 -24207,0 -24208,0 -24209,0 -24210,0 -24211,0 -24212,0 -24213,0 -24214,0 -24215,0 -24216,0 -24217,0 -24218,0 -24219,0 -24220,0 -24221,0 -24222,0 -24223,0 -24224,0 -24225,0 -24226,0 -24227,0 -24228,0 -24229,0 -24230,0 -24231,0 -24232,0 -24233,0 -24234,0 -24235,0 -24236,0 -24237,0 -24238,0 -24239,0 -24240,0 -24241,0 -24242,0 -24243,0 -24244,0 -24245,0 -24246,0 -24247,0 -24248,0 -24249,0 -24250,0 -24251,0 -24252,0 -24253,0 -24254,0 -24255,0 -24256,0 -24257,0 -24258,0 -24259,0 -24260,0 -24261,0 -24262,0 -24263,0 -24264,0 -24265,0 -24266,0 -24267,0 -24268,0 -24269,0 -24270,0 -24271,0 -24272,0 -24273,0 -24274,0 -24275,0 -24276,0 -24277,0 -24278,0 -24279,0 -24280,0 -24281,0 -24282,0 -24283,0 -24284,0 -24285,0 -24286,0 -24287,0 -24288,0 -24289,0 -24290,0 -24291,0 -24292,0 -24293,0 -24294,0 -24295,0 -24296,0 -24297,0 -24298,0 -24299,0 -24300,0 -24301,0 -24302,0 -24303,0 -24304,0 -24305,0 -24306,0 -24307,0 -24308,0 -24309,0 -24310,0 -24311,0 -24312,0 -24313,0 -24314,0 -24315,0 -24316,0 -24317,0 -24318,0 -24319,0 -24320,0 -24321,0 -24322,0 -24323,0 -24324,0 -24325,0 -24326,0 -24327,0 -24328,0 -24329,0 -24330,0 -24331,0 -24332,0 -24333,0 -24334,0 -24335,0 -24336,0 -24337,0 -24338,0 -24339,0 -24340,0 -24341,0 -24342,0 -24343,0 -24344,0 -24345,0 -24346,0 -24347,0 -24348,0 -24349,0 -24350,0 -24351,0 -24352,0 -24353,0 -24354,0 -24355,0 -24356,0 -24357,0 -24358,0 -24359,0 -24360,0 -24361,0 -24362,0 -24363,0 -24364,0 -24365,0 -24366,0 -24367,0 -24368,0 -24369,0 -24370,0 -24371,0 -24372,0 -24373,0 -24374,0 -24375,0 -24376,0 -24377,0 -24378,0 -24379,0 -24380,0 -24381,0 -24382,0 -24383,0 -24384,0 -24385,0 -24386,0 -24387,0 -24388,0 -24389,0 -24390,0 -24391,0 -24392,0 -24393,0 -24394,0 -24395,0 -24396,0 -24397,0 -24398,0 -24399,0 -24400,0 -24401,0 -24402,0 -24403,0 -24404,0 -24405,0 -24406,0 -24407,0 -24408,0 -24409,0 -24410,0 -24411,0 -24412,0 -24413,0 -24414,0 -24415,0 -24416,0 -24417,0 -24418,0 -24419,0 -24420,0 -24421,0 -24422,0 -24423,0 -24424,0 -24425,0 -24426,0 -24427,0 -24428,0 -24429,0 -24430,0 -24431,0 -24432,0 -24433,0 -24434,0 -24435,0 -24436,0 -24437,0 -24438,0 -24439,0 -24440,0 -24441,0 -24442,0 -24443,0 -24444,0 -24445,0 -24446,0 -24447,0 -24448,0 -24449,0 -24450,0 -24451,0 -24452,0 -24453,0 -24454,0 -24455,0 -24456,0 -24457,0 -24458,0 -24459,0 -24460,0 -24461,0 -24462,0 -24463,0 -24464,0 -24465,0 -24466,0 -24467,0 -24468,0 -24469,0 -24470,0 -24471,0 -24472,0 -24473,0 -24474,0 -24475,0 -24476,0 -24477,0 -24478,0 -24479,0 -24480,0 -24481,0 -24482,0 -24483,0 -24484,0 -24485,0 -24486,0 -24487,0 -24488,0 -24489,0 -24490,0 -24491,0 -24492,0 -24493,0 -24494,0 -24495,0 -24496,0 -24497,0 -24498,0 -24499,0 -24500,0 -24501,0 -24502,0 -24503,0 -24504,0 -24505,0 -24506,0 -24507,0 -24508,0 -24509,0 -24510,0 -24511,0 -24512,0 -24513,0 -24514,0 -24515,0 -24516,0 -24517,0 -24518,0 -24519,0 -24520,0 -24521,0 -24522,0 -24523,0 -24524,0 -24525,0 -24526,0 -24527,0 -24528,0 -24529,0 -24530,0 -24531,0 -24532,0 -24533,0 -24534,0 -24535,0 -24536,0 -24537,0 -24538,0 -24539,0 -24540,0 -24541,0 -24542,0 -24543,0 -24544,0 -24545,0 -24546,0 -24547,0 -24548,0 -24549,0 -24550,0 -24551,0 -24552,0 -24553,0 -24554,0 -24555,0 -24556,0 -24557,0 -24558,0 -24559,0 -24560,0 -24561,0 -24562,0 -24563,0 -24564,0 -24565,0 -24566,0 -24567,0 -24568,0 -24569,0 -24570,0 -24571,0 -24572,0 -24573,0 -24574,0 -24575,0 -24576,0 -24577,0 -24578,0 -24579,0 -24580,0 -24581,0 -24582,0 -24583,0 -24584,0 -24585,0 -24586,0 -24587,0 -24588,0 -24589,0 -24590,0 -24591,0 -24592,0 -24593,0 -24594,0 -24595,0 -24596,0 -24597,0 -24598,0 -24599,0 -24600,0 -24601,0 -24602,0 -24603,0 -24604,0 -24605,0 -24606,0 -24607,0 -24608,0 -24609,0 -24610,0 -24611,0 -24612,0 -24613,0 -24614,0 -24615,0 -24616,0 -24617,0 -24618,0 -24619,0 -24620,0 -24621,0 -24622,0 -24623,0 -24624,0 -24625,0 -24626,0 -24627,0 -24628,0 -24629,0 -24630,0 -24631,0 -24632,0 -24633,0 -24634,0 -24635,0 -24636,0 -24637,0 -24638,0 -24639,0 -24640,0 -24641,0 -24642,0 -24643,0 -24644,0 -24645,0 -24646,0 -24647,0 -24648,0 -24649,0 -24650,0 -24651,0 -24652,0 -24653,0 -24654,0 -24655,0 -24656,0 -24657,0 -24658,0 -24659,0 -24660,0 -24661,0 -24662,0 -24663,0 -24664,0 -24665,0 -24666,0 -24667,0 -24668,0 -24669,0 -24670,0 -24671,0 -24672,0 -24673,0 -24674,0 -24675,0 -24676,0 -24677,0 -24678,0 -24679,0 -24680,0 -24681,0 -24682,0 -24683,0 -24684,0 -24685,0 -24686,0 -24687,0 -24688,0 -24689,0 -24690,0 -24691,0 -24692,0 -24693,0 -24694,0 -24695,0 -24696,0 -24697,0 -24698,0 -24699,0 -24700,0 -24701,0 -24702,0 -24703,0 -24704,0 -24705,0 -24706,0 -24707,0 -24708,0 -24709,0 -24710,0 -24711,0 -24712,0 -24713,0 -24714,0 -24715,0 -24716,0 -24717,0 -24718,0 -24719,0 -24720,0 -24721,0 -24722,0 -24723,0 -24724,0 -24725,0 -24726,0 -24727,0 -24728,0 -24729,0 -24730,0 -24731,0 -24732,0 -24733,0 -24734,0 -24735,0 -24736,0 -24737,0 -24738,0 -24739,0 -24740,0 -24741,0 -24742,0 -24743,0 -24744,0 -24745,0 -24746,0 -24747,0 -24748,0 -24749,0 -24750,0 -24751,0 -24752,0 -24753,0 -24754,0 -24755,0 -24756,0 -24757,0 -24758,0 -24759,0 -24760,0 -24761,0 -24762,0 -24763,0 -24764,0 -24765,0 -24766,0 -24767,0 -24768,0 -24769,0 -24770,0 -24771,0 -24772,0 -24773,0 -24774,0 -24775,0 -24776,0 -24777,0 -24778,0 -24779,0 -24780,0 -24781,0 -24782,0 -24783,0 -24784,0 -24785,0 -24786,0 -24787,0 -24788,0 -24789,0 -24790,0 -24791,0 -24792,0 -24793,0 -24794,0 -24795,0 -24796,0 -24797,0 -24798,0 -24799,0 -24800,0 -24801,0 -24802,0 -24803,0 -24804,0 -24805,0 -24806,0 -24807,0 -24808,0 -24809,0 -24810,0 -24811,0 -24812,0 -24813,0 -24814,0 -24815,0 -24816,0 -24817,0 -24818,0 -24819,0 -24820,0 -24821,0 -24822,0 -24823,0 -24824,0 -24825,0 -24826,0 -24827,0 -24828,0 -24829,0 -24830,0 -24831,0 -24832,0 -24833,0 -24834,0 -24835,0 -24836,0 -24837,0 -24838,0 -24839,0 -24840,0 -24841,0 -24842,0 -24843,0 -24844,0 -24845,0 -24846,0 -24847,0 -24848,0 -24849,0 -24850,0 -24851,0 -24852,0 -24853,0 -24854,0 -24855,0 -24856,0 -24857,0 -24858,0 -24859,0 -24860,0 -24861,0 -24862,0 -24863,0 -24864,0 -24865,0 -24866,0 -24867,0 -24868,0 -24869,0 -24870,0 -24871,0 -24872,0 -24873,0 -24874,0 -24875,0 -24876,0 -24877,0 -24878,0 -24879,0 -24880,0 -24881,0 -24882,0 -24883,0 -24884,0 -24885,0 -24886,0 -24887,0 -24888,0 -24889,0 -24890,0 -24891,0 -24892,0 -24893,0 -24894,0 -24895,0 -24896,0 -24897,0 -24898,0 -24899,0 -24900,0 -24901,0 -24902,0 -24903,0 -24904,0 -24905,0 -24906,0 -24907,0 -24908,0 -24909,0 -24910,0 -24911,0 -24912,0 -24913,0 -24914,0 -24915,0 -24916,0 -24917,0 -24918,0 -24919,0 -24920,0 -24921,0 -24922,0 -24923,0 -24924,0 -24925,0 -24926,0 -24927,0 -24928,0 -24929,0 -24930,0 -24931,0 -24932,0 -24933,0 -24934,0 -24935,0 -24936,0 -24937,0 -24938,0 -24939,0 -24940,0 -24941,0 -24942,0 -24943,0 -24944,0 -24945,0 -24946,0 -24947,0 -24948,0 -24949,0 -24950,0 -24951,0 -24952,0 -24953,0 -24954,0 -24955,0 -24956,0 -24957,0 -24958,0 -24959,0 -24960,0 -24961,0 -24962,0 -24963,0 -24964,0 -24965,0 -24966,0 -24967,0 -24968,0 -24969,0 -24970,0 -24971,0 -24972,0 -24973,0 -24974,0 -24975,0 -24976,0 -24977,0 -24978,0 -24979,0 -24980,0 -24981,0 -24982,0 -24983,0 -24984,0 -24985,0 -24986,0 -24987,0 -24988,0 -24989,0 -24990,0 -24991,0 -24992,0 -24993,0 -24994,0 -24995,0 -24996,0 -24997,0 -24998,0 -24999,0 -25000,0 -25001,0 -25002,0 -25003,0 -25004,0 -25005,0 -25006,0 -25007,0 -25008,0 -25009,0 -25010,0 -25011,0 -25012,0 -25013,0 -25014,0 -25015,0 -25016,0 -25017,0 -25018,0 -25019,0 -25020,0 -25021,0 -25022,0 -25023,0 -25024,0 -25025,0 -25026,0 -25027,0 -25028,0 -25029,0 -25030,0 -25031,0 -25032,0 -25033,0 -25034,0 -25035,0 -25036,0 -25037,0 -25038,0 -25039,0 -25040,0 -25041,0 -25042,0 -25043,0 -25044,0 -25045,0 -25046,0 -25047,0 -25048,0 -25049,0 -25050,0 -25051,0 -25052,0 -25053,0 -25054,0 -25055,0 -25056,0 -25057,0 -25058,0 -25059,0 -25060,0 -25061,0 -25062,0 -25063,0 -25064,0 -25065,0 -25066,0 -25067,0 -25068,0 -25069,0 -25070,0 -25071,0 -25072,0 -25073,0 -25074,0 -25075,0 -25076,0 -25077,0 -25078,0 -25079,0 -25080,0 -25081,0 -25082,0 -25083,0 -25084,0 -25085,0 -25086,0 -25087,0 -25088,0 -25089,0 -25090,0 -25091,0 -25092,0 -25093,0 -25094,0 -25095,0 -25096,0 -25097,0 -25098,0 -25099,0 -25100,0 -25101,0 -25102,0 -25103,0 -25104,0 -25105,0 -25106,0 -25107,0 -25108,0 -25109,0 -25110,0 -25111,0 -25112,0 -25113,0 -25114,0 -25115,0 -25116,0 -25117,0 -25118,0 -25119,0 -25120,0 -25121,0 -25122,0 -25123,0 -25124,0 -25125,0 -25126,0 -25127,0 -25128,0 -25129,0 -25130,0 -25131,0 -25132,0 -25133,0 -25134,0 -25135,0 -25136,0 -25137,0 -25138,0 -25139,0 -25140,0 -25141,0 -25142,0 -25143,0 -25144,0 -25145,0 -25146,0 -25147,0 -25148,0 -25149,0 -25150,0 -25151,0 -25152,0 -25153,0 -25154,0 -25155,0 -25156,0 -25157,0 -25158,0 -25159,0 -25160,0 -25161,0 -25162,0 -25163,0 -25164,0 -25165,0 -25166,0 -25167,0 -25168,0 -25169,0 -25170,0 -25171,0 -25172,0 -25173,0 -25174,0 -25175,0 -25176,0 -25177,0 -25178,0 -25179,0 -25180,0 -25181,0 -25182,0 -25183,0 -25184,0 -25185,0 -25186,0 -25187,0 -25188,0 -25189,0 -25190,0 -25191,0 -25192,0 -25193,0 -25194,0 -25195,0 -25196,0 -25197,0 -25198,0 -25199,0 -25200,0 -25201,0 -25202,0 -25203,0 -25204,0 -25205,0 -25206,0 -25207,0 -25208,0 -25209,0 -25210,0 -25211,0 -25212,0 -25213,0 -25214,0 -25215,0 -25216,0 -25217,0 -25218,0 -25219,0 -25220,0 -25221,0 -25222,0 -25223,0 -25224,0 -25225,0 -25226,0 -25227,0 -25228,0 -25229,0 -25230,0 -25231,0 -25232,0 -25233,0 -25234,0 -25235,0 -25236,0 -25237,0 -25238,0 -25239,0 -25240,0 -25241,0 -25242,0 -25243,0 -25244,0 -25245,0 -25246,0 -25247,0 -25248,0 -25249,0 -25250,0 -25251,0 -25252,0 -25253,0 -25254,0 -25255,0 -25256,0 -25257,0 -25258,0 -25259,0 -25260,0 -25261,0 -25262,0 -25263,0 -25264,0 -25265,0 -25266,0 -25267,0 -25268,0 -25269,0 -25270,0 -25271,0 -25272,0 -25273,0 -25274,0 -25275,0 -25276,0 -25277,0 -25278,0 -25279,0 -25280,0 -25281,0 -25282,0 -25283,0 -25284,0 -25285,0 -25286,0 -25287,0 -25288,0 -25289,0 -25290,0 -25291,0 -25292,0 -25293,0 -25294,0 -25295,0 -25296,0 -25297,0 -25298,0 -25299,0 -25300,0 -25301,0 -25302,0 -25303,0 -25304,0 -25305,0 -25306,0 -25307,0 -25308,0 -25309,0 -25310,0 -25311,0 -25312,0 -25313,0 -25314,0 -25315,0 -25316,0 -25317,0 -25318,0 -25319,0 -25320,0 -25321,0 -25322,0 -25323,0 -25324,0 -25325,0 -25326,0 -25327,0 -25328,0 -25329,0 -25330,0 -25331,0 -25332,0 -25333,0 -25334,0 -25335,0 -25336,0 -25337,0 -25338,0 -25339,0 -25340,0 -25341,0 -25342,0 -25343,0 -25344,0 -25345,0 -25346,0 -25347,0 -25348,0 -25349,0 -25350,0 -25351,0 -25352,0 -25353,0 -25354,0 -25355,0 -25356,0 -25357,0 -25358,0 -25359,0 -25360,0 -25361,0 -25362,0 -25363,0 -25364,0 -25365,0 -25366,0 -25367,0 -25368,0 -25369,0 -25370,0 -25371,0 -25372,0 -25373,0 -25374,0 -25375,0 -25376,0 -25377,0 -25378,0 -25379,0 -25380,0 -25381,0 -25382,0 -25383,0 -25384,0 -25385,0 -25386,0 -25387,0 -25388,0 -25389,0 -25390,0 -25391,0 -25392,0 -25393,0 -25394,0 -25395,0 -25396,0 -25397,0 -25398,0 -25399,0 -25400,0 -25401,0 -25402,0 -25403,0 -25404,0 -25405,0 -25406,0 -25407,0 -25408,0 -25409,0 -25410,0 -25411,0 -25412,0 -25413,0 -25414,0 -25415,0 -25416,0 -25417,0 -25418,0 -25419,0 -25420,0 -25421,0 -25422,0 -25423,0 -25424,0 -25425,0 -25426,0 -25427,0 -25428,0 -25429,0 -25430,0 -25431,0 -25432,0 -25433,0 -25434,0 -25435,0 -25436,0 -25437,0 -25438,0 -25439,0 -25440,0 -25441,0 -25442,0 -25443,0 -25444,0 -25445,0 -25446,0 -25447,0 -25448,0 -25449,0 -25450,0 -25451,0 -25452,0 -25453,0 -25454,0 -25455,0 -25456,0 -25457,0 -25458,0 -25459,0 -25460,0 -25461,0 -25462,0 -25463,0 -25464,0 -25465,0 -25466,0 -25467,0 -25468,0 -25469,0 -25470,0 -25471,0 -25472,0 -25473,0 -25474,0 -25475,0 -25476,0 -25477,0 -25478,0 -25479,0 -25480,0 -25481,0 -25482,0 -25483,0 -25484,0 -25485,0 -25486,0 -25487,0 -25488,0 -25489,0 -25490,0 -25491,0 -25492,0 -25493,0 -25494,0 -25495,0 -25496,0 -25497,0 -25498,0 -25499,0 -25500,0 -25501,0 -25502,0 -25503,0 -25504,0 -25505,0 -25506,0 -25507,0 -25508,0 -25509,0 -25510,0 -25511,0 -25512,0 -25513,0 -25514,0 -25515,0 -25516,0 -25517,0 -25518,0 -25519,0 -25520,0 -25521,0 -25522,0 -25523,0 -25524,0 -25525,0 -25526,0 -25527,0 -25528,0 -25529,0 -25530,0 -25531,0 -25532,0 -25533,0 -25534,0 -25535,0 -25536,0 -25537,0 -25538,0 -25539,0 -25540,0 -25541,0 -25542,0 -25543,0 -25544,0 -25545,0 -25546,0 -25547,0 -25548,0 -25549,0 -25550,0 -25551,0 -25552,0 -25553,0 -25554,0 -25555,0 -25556,0 -25557,0 -25558,0 -25559,0 -25560,0 -25561,0 -25562,0 -25563,0 -25564,0 -25565,0 -25566,0 -25567,0 -25568,0 -25569,0 -25570,0 -25571,0 -25572,0 -25573,0 -25574,0 -25575,0 -25576,0 -25577,0 -25578,0 -25579,0 -25580,0 -25581,0 -25582,0 -25583,0 -25584,0 -25585,0 -25586,0 -25587,0 -25588,0 -25589,0 -25590,0 -25591,0 -25592,0 -25593,0 -25594,0 -25595,0 -25596,0 -25597,0 -25598,0 -25599,0 -25600,0 -25601,0 -25602,0 -25603,0 -25604,0 -25605,0 -25606,0 -25607,0 -25608,0 -25609,0 -25610,0 -25611,0 -25612,0 -25613,0 -25614,0 -25615,0 -25616,0 -25617,0 -25618,0 -25619,0 -25620,0 -25621,0 -25622,0 -25623,0 -25624,0 -25625,0 -25626,0 -25627,0 -25628,0 -25629,0 -25630,0 -25631,0 -25632,0 -25633,0 -25634,0 -25635,0 -25636,0 -25637,0 -25638,0 -25639,0 -25640,0 -25641,0 -25642,0 -25643,0 -25644,0 -25645,0 -25646,0 -25647,0 -25648,0 -25649,0 -25650,0 -25651,0 -25652,0 -25653,0 -25654,0 -25655,0 -25656,0 -25657,0 -25658,0 -25659,0 -25660,0 -25661,0 -25662,0 -25663,0 -25664,0 -25665,0 -25666,0 -25667,0 -25668,0 -25669,0 -25670,0 -25671,0 -25672,0 -25673,0 -25674,0 -25675,0 -25676,0 -25677,0 -25678,0 -25679,0 -25680,0 -25681,0 -25682,0 -25683,0 -25684,0 -25685,0 -25686,0 -25687,0 -25688,0 -25689,0 -25690,0 -25691,0 -25692,0 -25693,0 -25694,0 -25695,0 -25696,0 -25697,0 -25698,0 -25699,0 -25700,0 -25701,0 -25702,0 -25703,0 -25704,0 -25705,0 -25706,0 -25707,0 -25708,0 -25709,0 -25710,0 -25711,0 -25712,0 -25713,0 -25714,0 -25715,0 -25716,0 -25717,0 -25718,0 -25719,0 -25720,0 -25721,0 -25722,0 -25723,0 -25724,0 -25725,0 -25726,0 -25727,0 -25728,0 -25729,0 -25730,0 -25731,0 -25732,0 -25733,0 -25734,0 -25735,0 -25736,0 -25737,0 -25738,0 -25739,0 -25740,0 -25741,0 -25742,0 -25743,0 -25744,0 -25745,0 -25746,0 -25747,0 -25748,0 -25749,0 -25750,0 -25751,0 -25752,0 -25753,0 -25754,0 -25755,0 -25756,0 -25757,0 -25758,0 -25759,0 -25760,0 -25761,0 -25762,0 -25763,0 -25764,0 -25765,0 -25766,0 -25767,0 -25768,0 -25769,0 -25770,0 -25771,0 -25772,0 -25773,0 -25774,0 -25775,0 -25776,0 -25777,0 -25778,0 -25779,0 -25780,0 -25781,0 -25782,0 -25783,0 -25784,0 -25785,0 -25786,0 -25787,0 -25788,0 -25789,0 -25790,0 -25791,0 -25792,0 -25793,0 -25794,0 -25795,0 -25796,0 -25797,0 -25798,0 -25799,0 -25800,0 -25801,0 -25802,0 -25803,0 -25804,0 -25805,0 -25806,0 -25807,0 -25808,0 -25809,0 -25810,0 -25811,0 -25812,0 -25813,0 -25814,0 -25815,0 -25816,0 -25817,0 -25818,0 -25819,0 -25820,0 -25821,0 -25822,0 -25823,0 -25824,0 -25825,0 -25826,0 -25827,0 -25828,0 -25829,0 -25830,0 -25831,0 -25832,0 -25833,0 -25834,0 -25835,0 -25836,0 -25837,0 -25838,0 -25839,0 -25840,0 -25841,0 -25842,0 -25843,0 -25844,0 -25845,0 -25846,0 -25847,0 -25848,0 -25849,0 -25850,0 -25851,0 -25852,0 -25853,0 -25854,0 -25855,0 -25856,0 -25857,0 -25858,0 -25859,0 -25860,0 -25861,0 -25862,0 -25863,0 -25864,0 -25865,0 -25866,0 -25867,0 -25868,0 -25869,0 -25870,0 -25871,0 -25872,0 -25873,0 -25874,0 -25875,0 -25876,0 -25877,0 -25878,0 -25879,0 -25880,0 -25881,0 -25882,0 -25883,0 -25884,0 -25885,0 -25886,0 -25887,0 -25888,0 -25889,0 -25890,0 -25891,0 -25892,0 -25893,0 -25894,0 -25895,0 -25896,0 -25897,0 -25898,0 -25899,0 -25900,0 -25901,0 -25902,0 -25903,0 -25904,0 -25905,0 -25906,0 -25907,0 -25908,0 -25909,0 -25910,0 -25911,0 -25912,0 -25913,0 -25914,0 -25915,0 -25916,0 -25917,0 -25918,0 -25919,0 -25920,0 -25921,0 -25922,0 -25923,0 -25924,0 -25925,0 -25926,0 -25927,0 -25928,0 -25929,0 -25930,0 -25931,0 -25932,0 -25933,0 -25934,0 -25935,0 -25936,0 -25937,0 -25938,0 -25939,0 -25940,0 -25941,0 -25942,0 -25943,0 -25944,0 -25945,0 -25946,0 -25947,0 -25948,0 -25949,0 -25950,0 -25951,0 -25952,0 -25953,0 -25954,0 -25955,0 -25956,0 -25957,0 -25958,0 -25959,0 -25960,0 -25961,0 -25962,0 -25963,0 -25964,0 -25965,0 -25966,0 -25967,0 -25968,0 -25969,0 -25970,0 -25971,0 -25972,0 -25973,0 -25974,0 -25975,0 -25976,0 -25977,0 -25978,0 -25979,0 -25980,0 -25981,0 -25982,0 -25983,0 -25984,0 -25985,0 -25986,0 -25987,0 -25988,0 -25989,0 -25990,0 -25991,0 -25992,0 -25993,0 -25994,0 -25995,0 -25996,0 -25997,0 -25998,0 -25999,0 -26000,0 -26001,0 -26002,0 -26003,0 -26004,0 -26005,0 -26006,0 -26007,0 -26008,0 -26009,0 -26010,0 -26011,0 -26012,0 -26013,0 -26014,0 -26015,0 -26016,0 -26017,0 -26018,0 -26019,0 -26020,0 -26021,0 -26022,0 -26023,0 -26024,0 -26025,0 -26026,0 -26027,0 -26028,0 -26029,0 -26030,0 -26031,0 -26032,0 -26033,0 -26034,0 -26035,0 -26036,0 -26037,0 -26038,0 -26039,0 -26040,0 -26041,0 -26042,0 -26043,0 -26044,0 -26045,0 -26046,0 -26047,0 -26048,0 -26049,0 -26050,0 -26051,0 -26052,0 -26053,0 -26054,0 -26055,0 -26056,0 -26057,0 -26058,0 -26059,0 -26060,0 -26061,0 -26062,0 -26063,0 -26064,0 -26065,0 -26066,0 -26067,0 -26068,0 -26069,0 -26070,0 -26071,0 -26072,0 -26073,0 -26074,0 -26075,0 -26076,0 -26077,0 -26078,0 -26079,0 -26080,0 -26081,0 -26082,0 -26083,0 -26084,0 -26085,0 -26086,0 -26087,0 -26088,0 -26089,0 -26090,0 -26091,0 -26092,0 -26093,0 -26094,0 -26095,0 -26096,0 -26097,0 -26098,0 -26099,0 -26100,0 -26101,0 -26102,0 -26103,0 -26104,0 -26105,0 -26106,0 -26107,0 -26108,0 -26109,0 -26110,0 -26111,0 -26112,0 -26113,0 -26114,0 -26115,0 -26116,0 -26117,0 -26118,0 -26119,0 -26120,0 -26121,0 -26122,0 -26123,0 -26124,0 -26125,0 -26126,0 -26127,0 -26128,0 -26129,0 -26130,0 -26131,0 -26132,0 -26133,0 -26134,0 -26135,0 -26136,0 -26137,0 -26138,0 -26139,0 -26140,0 -26141,0 -26142,0 -26143,0 -26144,0 -26145,0 -26146,0 -26147,0 -26148,0 -26149,0 -26150,0 -26151,0 -26152,0 -26153,0 -26154,0 -26155,0 -26156,0 -26157,0 -26158,0 -26159,0 -26160,0 -26161,0 -26162,0 -26163,0 -26164,0 -26165,0 -26166,0 -26167,0 -26168,0 -26169,0 -26170,0 -26171,0 -26172,0 -26173,0 -26174,0 -26175,0 -26176,0 -26177,0 -26178,0 -26179,0 -26180,0 -26181,0 -26182,0 -26183,0 -26184,0 -26185,0 -26186,0 -26187,0 -26188,0 -26189,0 -26190,0 -26191,0 -26192,0 -26193,0 -26194,0 -26195,0 -26196,0 -26197,0 -26198,0 -26199,0 -26200,0 -26201,0 -26202,0 -26203,0 -26204,0 -26205,0 -26206,0 -26207,0 -26208,0 -26209,0 -26210,0 -26211,0 -26212,0 -26213,0 -26214,0 -26215,0 -26216,0 -26217,0 -26218,0 -26219,0 -26220,0 -26221,0 -26222,0 -26223,0 -26224,0 -26225,0 -26226,0 -26227,0 -26228,0 -26229,0 -26230,0 -26231,0 -26232,0 -26233,0 -26234,0 -26235,0 -26236,0 -26237,0 -26238,0 -26239,0 -26240,0 -26241,0 -26242,0 -26243,0 -26244,0 -26245,0 -26246,0 -26247,0 -26248,0 -26249,0 -26250,0 -26251,0 -26252,0 -26253,0 -26254,0 -26255,0 -26256,0 -26257,0 -26258,0 -26259,0 -26260,0 -26261,0 -26262,0 -26263,0 -26264,0 -26265,0 -26266,0 -26267,0 -26268,0 -26269,0 -26270,0 -26271,0 -26272,0 -26273,0 -26274,0 -26275,0 -26276,0 -26277,0 -26278,0 -26279,0 -26280,0 -26281,0 -26282,0 -26283,0 -26284,0 -26285,0 -26286,0 -26287,0 -26288,0 -26289,0 -26290,0 -26291,0 -26292,0 -26293,0 -26294,0 -26295,0 -26296,0 -26297,0 -26298,0 -26299,0 -26300,0 -26301,0 -26302,0 -26303,0 -26304,0 -26305,0 -26306,0 -26307,0 -26308,0 -26309,0 -26310,0 -26311,0 -26312,0 -26313,0 -26314,0 -26315,0 -26316,0 -26317,0 -26318,0 -26319,0 -26320,0 -26321,0 -26322,0 -26323,0 -26324,0 -26325,0 -26326,0 -26327,0 -26328,0 -26329,0 -26330,0 -26331,0 -26332,0 -26333,0 -26334,0 -26335,0 -26336,0 -26337,0 -26338,0 -26339,0 -26340,0 -26341,0 -26342,0 -26343,0 -26344,0 -26345,0 -26346,0 -26347,0 -26348,0 -26349,0 -26350,0 -26351,0 -26352,0 -26353,0 -26354,0 -26355,0 -26356,0 -26357,0 -26358,0 -26359,0 -26360,0 -26361,0 -26362,0 -26363,0 -26364,0 -26365,0 -26366,0 -26367,0 -26368,0 -26369,0 -26370,0 -26371,0 -26372,0 -26373,0 -26374,0 -26375,0 -26376,0 -26377,0 -26378,0 -26379,0 -26380,0 -26381,0 -26382,0 -26383,0 -26384,0 -26385,0 -26386,0 -26387,0 -26388,0 -26389,0 -26390,0 -26391,0 -26392,0 -26393,0 -26394,0 -26395,0 -26396,0 -26397,0 -26398,0 -26399,0 -26400,0 -26401,0 -26402,0 -26403,0 -26404,0 -26405,0 -26406,0 -26407,0 -26408,0 -26409,0 -26410,0 -26411,0 -26412,0 -26413,0 -26414,0 -26415,0 -26416,0 -26417,0 -26418,0 -26419,0 -26420,0 -26421,0 -26422,0 -26423,0 -26424,0 -26425,0 -26426,0 -26427,0 -26428,0 -26429,0 -26430,0 -26431,0 -26432,0 -26433,0 -26434,0 -26435,0 -26436,0 -26437,0 -26438,0 -26439,0 -26440,0 -26441,0 -26442,0 -26443,0 -26444,0 -26445,0 -26446,0 -26447,0 -26448,0 -26449,0 -26450,0 -26451,0 -26452,0 -26453,0 -26454,0 -26455,0 -26456,0 -26457,0 -26458,0 -26459,0 -26460,0 -26461,0 -26462,0 -26463,0 -26464,0 -26465,0 -26466,0 -26467,0 -26468,0 -26469,0 -26470,0 -26471,0 -26472,0 -26473,0 -26474,0 -26475,0 -26476,0 -26477,0 -26478,0 -26479,0 -26480,0 -26481,0 -26482,0 -26483,0 -26484,0 -26485,0 -26486,0 -26487,0 -26488,0 -26489,0 -26490,0 -26491,0 -26492,0 -26493,0 -26494,0 -26495,0 -26496,0 -26497,0 -26498,0 -26499,0 -26500,0 -26501,0 -26502,0 -26503,0 -26504,0 -26505,0 -26506,0 -26507,0 -26508,0 -26509,0 -26510,0 -26511,0 -26512,0 -26513,0 -26514,0 -26515,0 -26516,0 -26517,0 -26518,0 -26519,0 -26520,0 -26521,0 -26522,0 -26523,0 -26524,0 -26525,0 -26526,0 -26527,0 -26528,0 -26529,0 -26530,0 -26531,0 -26532,0 -26533,0 -26534,0 -26535,0 -26536,0 -26537,0 -26538,0 -26539,0 -26540,0 -26541,0 -26542,0 -26543,0 -26544,0 -26545,0 -26546,0 -26547,0 -26548,0 -26549,0 -26550,0 -26551,0 -26552,0 -26553,0 -26554,0 -26555,0 -26556,0 -26557,0 -26558,0 -26559,0 -26560,0 -26561,0 -26562,0 -26563,0 -26564,0 -26565,0 -26566,0 -26567,0 -26568,0 -26569,0 -26570,0 -26571,0 -26572,0 -26573,0 -26574,0 -26575,0 -26576,0 -26577,0 -26578,0 -26579,0 -26580,0 -26581,0 -26582,0 -26583,0 -26584,0 -26585,0 -26586,0 -26587,0 -26588,0 -26589,0 -26590,0 -26591,0 -26592,0 -26593,0 -26594,0 -26595,0 -26596,0 -26597,0 -26598,0 -26599,0 -26600,0 -26601,0 -26602,0 -26603,0 -26604,0 -26605,0 -26606,0 -26607,0 -26608,0 -26609,0 -26610,0 -26611,0 -26612,0 -26613,0 -26614,0 -26615,0 -26616,0 -26617,0 -26618,0 -26619,0 -26620,0 -26621,0 -26622,0 -26623,0 -26624,0 -26625,0 -26626,0 -26627,0 -26628,0 -26629,0 -26630,0 -26631,0 -26632,0 -26633,0 -26634,0 -26635,0 -26636,0 -26637,0 -26638,0 -26639,0 -26640,0 -26641,0 -26642,0 -26643,0 -26644,0 -26645,0 -26646,0 -26647,0 -26648,0 -26649,0 -26650,0 -26651,0 -26652,0 -26653,0 -26654,0 -26655,0 -26656,0 -26657,0 -26658,0 -26659,0 -26660,0 -26661,0 -26662,0 -26663,0 -26664,0 -26665,0 -26666,0 -26667,0 -26668,0 -26669,0 -26670,0 -26671,0 -26672,0 -26673,0 -26674,0 -26675,0 -26676,0 -26677,0 -26678,0 -26679,0 -26680,0 -26681,0 -26682,0 -26683,0 -26684,0 -26685,0 -26686,0 -26687,0 -26688,0 -26689,0 -26690,0 -26691,0 -26692,0 -26693,0 -26694,0 -26695,0 -26696,0 -26697,0 -26698,0 -26699,0 -26700,0 -26701,0 -26702,0 -26703,0 -26704,0 -26705,0 -26706,0 -26707,0 -26708,0 -26709,0 -26710,0 -26711,0 -26712,0 -26713,0 -26714,0 -26715,0 -26716,0 -26717,0 -26718,0 -26719,0 -26720,0 -26721,0 -26722,0 -26723,0 -26724,0 -26725,0 -26726,0 -26727,0 -26728,0 -26729,0 -26730,0 -26731,0 -26732,0 -26733,0 -26734,0 -26735,0 -26736,0 -26737,0 -26738,0 -26739,0 -26740,0 -26741,0 -26742,0 -26743,0 -26744,0 -26745,0 -26746,0 -26747,0 -26748,0 -26749,0 -26750,0 -26751,0 -26752,0 -26753,0 -26754,0 -26755,0 -26756,0 -26757,0 -26758,0 -26759,0 -26760,0 -26761,0 -26762,0 -26763,0 -26764,0 -26765,0 -26766,0 -26767,0 -26768,0 -26769,0 -26770,0 -26771,0 -26772,0 -26773,0 -26774,0 -26775,0 -26776,0 -26777,0 -26778,0 -26779,0 -26780,0 -26781,0 -26782,0 -26783,0 -26784,0 -26785,0 -26786,0 -26787,0 -26788,0 -26789,0 -26790,0 -26791,0 -26792,0 -26793,0 -26794,0 -26795,0 -26796,0 -26797,0 -26798,0 -26799,0 -26800,0 -26801,0 -26802,0 -26803,0 -26804,0 -26805,0 -26806,0 -26807,0 -26808,0 -26809,0 -26810,0 -26811,0 -26812,0 -26813,0 -26814,0 -26815,0 -26816,0 -26817,0 -26818,0 -26819,0 -26820,0 -26821,0 -26822,0 -26823,0 -26824,0 -26825,0 -26826,0 -26827,0 -26828,0 -26829,0 -26830,0 -26831,0 -26832,0 -26833,0 -26834,0 -26835,0 -26836,0 -26837,0 -26838,0 -26839,0 -26840,0 -26841,0 -26842,0 -26843,0 -26844,0 -26845,0 -26846,0 -26847,0 -26848,0 -26849,0 -26850,0 -26851,0 -26852,0 -26853,0 -26854,0 -26855,0 -26856,0 -26857,0 -26858,0 -26859,0 -26860,0 -26861,0 -26862,0 -26863,0 -26864,0 -26865,0 -26866,0 -26867,0 -26868,0 -26869,0 -26870,0 -26871,0 -26872,0 -26873,0 -26874,0 -26875,0 -26876,0 -26877,0 -26878,0 -26879,0 -26880,0 -26881,0 -26882,0 -26883,0 -26884,0 -26885,0 -26886,0 -26887,0 -26888,0 -26889,0 -26890,0 -26891,0 -26892,0 -26893,0 -26894,0 -26895,0 -26896,0 -26897,0 -26898,0 -26899,0 -26900,0 -26901,0 -26902,0 -26903,0 -26904,0 -26905,0 -26906,0 -26907,0 -26908,0 -26909,0 -26910,0 -26911,0 -26912,0 -26913,0 -26914,0 -26915,0 -26916,0 -26917,0 -26918,0 -26919,0 -26920,0 -26921,0 -26922,0 -26923,0 -26924,0 -26925,0 -26926,0 -26927,0 -26928,0 -26929,0 -26930,0 -26931,0 -26932,0 -26933,0 -26934,0 -26935,0 -26936,0 -26937,0 -26938,0 -26939,0 -26940,0 -26941,0 -26942,0 -26943,0 -26944,0 -26945,0 -26946,0 -26947,0 -26948,0 -26949,0 -26950,0 -26951,0 -26952,0 -26953,0 -26954,0 -26955,0 -26956,0 -26957,0 -26958,0 -26959,0 -26960,0 -26961,0 -26962,0 -26963,0 -26964,0 -26965,0 -26966,0 -26967,0 -26968,0 -26969,0 -26970,0 -26971,0 -26972,0 -26973,0 -26974,0 -26975,0 -26976,0 -26977,0 -26978,0 -26979,0 -26980,0 -26981,0 -26982,0 -26983,0 -26984,0 -26985,0 -26986,0 -26987,0 -26988,0 -26989,0 -26990,0 -26991,0 -26992,0 -26993,0 -26994,0 -26995,0 -26996,0 -26997,0 -26998,0 -26999,0 -27000,0 -27001,0 -27002,0 -27003,0 -27004,0 -27005,0 -27006,0 -27007,0 -27008,0 -27009,0 -27010,0 -27011,0 -27012,0 -27013,0 -27014,0 -27015,0 -27016,0 -27017,0 -27018,0 -27019,0 -27020,0 -27021,0 -27022,0 -27023,0 -27024,0 -27025,0 -27026,0 -27027,0 -27028,0 -27029,0 -27030,0 -27031,0 -27032,0 -27033,0 -27034,0 -27035,0 -27036,0 -27037,0 -27038,0 -27039,0 -27040,0 -27041,0 -27042,0 -27043,0 -27044,0 -27045,0 -27046,0 -27047,0 -27048,0 -27049,0 -27050,0 -27051,0 -27052,0 -27053,0 -27054,0 -27055,0 -27056,0 -27057,0 -27058,0 -27059,0 -27060,0 -27061,0 -27062,0 -27063,0 -27064,0 -27065,0 -27066,0 -27067,0 -27068,0 -27069,0 -27070,0 -27071,0 -27072,0 -27073,0 -27074,0 -27075,0 -27076,0 -27077,0 -27078,0 -27079,0 -27080,0 -27081,0 -27082,0 -27083,0 -27084,0 -27085,0 -27086,0 -27087,0 -27088,0 -27089,0 -27090,0 -27091,0 -27092,0 -27093,0 -27094,0 -27095,0 -27096,0 -27097,0 -27098,0 -27099,0 -27100,0 -27101,0 -27102,0 -27103,0 -27104,0 -27105,0 -27106,0 -27107,0 -27108,0 -27109,0 -27110,0 -27111,0 -27112,0 -27113,0 -27114,0 -27115,0 -27116,0 -27117,0 -27118,0 -27119,0 -27120,0 -27121,0 -27122,0 -27123,0 -27124,0 -27125,0 -27126,0 -27127,0 -27128,0 -27129,0 -27130,0 -27131,0 -27132,0 -27133,0 -27134,0 -27135,0 -27136,0 -27137,0 -27138,0 -27139,0 -27140,0 -27141,0 -27142,0 -27143,0 -27144,0 -27145,0 -27146,0 -27147,0 -27148,0 -27149,0 -27150,0 -27151,0 -27152,0 -27153,0 -27154,0 -27155,0 -27156,0 -27157,0 -27158,0 -27159,0 -27160,0 -27161,0 -27162,0 -27163,0 -27164,0 -27165,0 -27166,0 -27167,0 -27168,0 -27169,0 -27170,0 -27171,0 -27172,0 -27173,0 -27174,0 -27175,0 -27176,0 -27177,0 -27178,0 -27179,0 -27180,0 -27181,0 -27182,0 -27183,0 -27184,0 -27185,0 -27186,0 -27187,0 -27188,0 -27189,0 -27190,0 -27191,0 -27192,0 -27193,0 -27194,0 -27195,0 -27196,0 -27197,0 -27198,0 -27199,0 -27200,0 -27201,0 -27202,0 -27203,0 -27204,0 -27205,0 -27206,0 -27207,0 -27208,0 -27209,0 -27210,0 -27211,0 -27212,0 -27213,0 -27214,0 -27215,0 -27216,0 -27217,0 -27218,0 -27219,0 -27220,0 -27221,0 -27222,0 -27223,0 -27224,0 -27225,0 -27226,0 -27227,0 -27228,0 -27229,0 -27230,0 -27231,0 -27232,0 -27233,0 -27234,0 -27235,0 -27236,0 -27237,0 -27238,0 -27239,0 -27240,0 -27241,0 -27242,0 -27243,0 -27244,0 -27245,0 -27246,0 -27247,0 -27248,0 -27249,0 -27250,0 -27251,0 -27252,0 -27253,0 -27254,0 -27255,0 -27256,0 -27257,0 -27258,0 -27259,0 -27260,0 -27261,0 -27262,0 -27263,0 -27264,0 -27265,0 -27266,0 -27267,0 -27268,0 -27269,0 -27270,0 -27271,0 -27272,0 -27273,0 -27274,0 -27275,0 -27276,0 -27277,0 -27278,0 -27279,0 -27280,0 -27281,0 -27282,0 -27283,0 -27284,0 -27285,0 -27286,0 -27287,0 -27288,0 -27289,0 -27290,0 -27291,0 -27292,0 -27293,0 -27294,0 -27295,0 -27296,0 -27297,0 -27298,0 -27299,0 -27300,0 -27301,0 -27302,0 -27303,0 -27304,0 -27305,0 -27306,0 -27307,0 -27308,0 -27309,0 -27310,0 -27311,0 -27312,0 -27313,0 -27314,0 -27315,0 -27316,0 -27317,0 -27318,0 -27319,0 -27320,0 -27321,0 -27322,0 -27323,0 -27324,0 -27325,0 -27326,0 -27327,0 -27328,0 -27329,0 -27330,0 -27331,0 -27332,0 -27333,0 -27334,0 -27335,0 -27336,0 -27337,0 -27338,0 -27339,0 -27340,0 -27341,0 -27342,0 -27343,0 -27344,0 -27345,0 -27346,0 -27347,0 -27348,0 -27349,0 -27350,0 -27351,0 -27352,0 -27353,0 -27354,0 -27355,0 -27356,0 -27357,0 -27358,0 -27359,0 -27360,0 -27361,0 -27362,0 -27363,0 -27364,0 -27365,0 -27366,0 -27367,0 -27368,0 -27369,0 -27370,0 -27371,0 -27372,0 -27373,0 -27374,0 -27375,0 -27376,0 -27377,0 -27378,0 -27379,0 -27380,0 -27381,0 -27382,0 -27383,0 -27384,0 -27385,0 -27386,0 -27387,0 -27388,0 -27389,0 -27390,0 -27391,0 -27392,0 -27393,0 -27394,0 -27395,0 -27396,0 -27397,0 -27398,0 -27399,0 -27400,0 -27401,0 -27402,0 -27403,0 -27404,0 -27405,0 -27406,0 -27407,0 -27408,0 -27409,0 -27410,0 -27411,0 -27412,0 -27413,0 -27414,0 -27415,0 -27416,0 -27417,0 -27418,0 -27419,0 -27420,0 -27421,0 -27422,0 -27423,0 -27424,0 -27425,0 -27426,0 -27427,0 -27428,0 -27429,0 -27430,0 -27431,0 -27432,0 -27433,0 -27434,0 -27435,0 -27436,0 -27437,0 -27438,0 -27439,0 -27440,0 -27441,0 -27442,0 -27443,0 -27444,0 -27445,0 -27446,0 -27447,0 -27448,0 -27449,0 -27450,0 -27451,0 -27452,0 -27453,0 -27454,0 -27455,0 -27456,0 -27457,0 -27458,0 -27459,0 -27460,0 -27461,0 -27462,0 -27463,0 -27464,0 -27465,0 -27466,0 -27467,0 -27468,0 -27469,0 -27470,0 -27471,0 -27472,0 -27473,0 -27474,0 -27475,0 -27476,0 -27477,0 -27478,0 -27479,0 -27480,0 -27481,0 -27482,0 -27483,0 -27484,0 -27485,0 -27486,0 -27487,0 -27488,0 -27489,0 -27490,0 -27491,0 -27492,0 -27493,0 -27494,0 -27495,0 -27496,0 -27497,0 -27498,0 -27499,0 -27500,0 -27501,0 -27502,0 -27503,0 -27504,0 -27505,0 -27506,0 -27507,0 -27508,0 -27509,0 -27510,0 -27511,0 -27512,0 -27513,0 -27514,0 -27515,0 -27516,0 -27517,0 -27518,0 -27519,0 -27520,0 -27521,0 -27522,0 -27523,0 -27524,0 -27525,0 -27526,0 -27527,0 -27528,0 -27529,0 -27530,0 -27531,0 -27532,0 -27533,0 -27534,0 -27535,0 -27536,0 -27537,0 -27538,0 -27539,0 -27540,0 -27541,0 -27542,0 -27543,0 -27544,0 -27545,0 -27546,0 -27547,0 -27548,0 -27549,0 -27550,0 -27551,0 -27552,0 -27553,0 -27554,0 -27555,0 -27556,0 -27557,0 -27558,0 -27559,0 -27560,0 -27561,0 -27562,0 -27563,0 -27564,0 -27565,0 -27566,0 -27567,0 -27568,0 -27569,0 -27570,0 -27571,0 -27572,0 -27573,0 -27574,0 -27575,0 -27576,0 -27577,0 -27578,0 -27579,0 -27580,0 -27581,0 -27582,0 -27583,0 -27584,0 -27585,0 -27586,0 -27587,0 -27588,0 -27589,0 -27590,0 -27591,0 -27592,0 -27593,0 -27594,0 -27595,0 -27596,0 -27597,0 -27598,0 -27599,0 -27600,0 -27601,0 -27602,0 -27603,0 -27604,0 -27605,0 -27606,0 -27607,0 -27608,0 -27609,0 -27610,0 -27611,0 -27612,0 -27613,0 -27614,0 -27615,0 -27616,0 -27617,0 -27618,0 -27619,0 -27620,0 -27621,0 -27622,0 -27623,0 -27624,0 -27625,0 -27626,0 -27627,0 -27628,0 -27629,0 -27630,0 -27631,0 -27632,0 -27633,0 -27634,0 -27635,0 -27636,0 -27637,0 -27638,0 -27639,0 -27640,0 -27641,0 -27642,0 -27643,0 -27644,0 -27645,0 -27646,0 -27647,0 -27648,0 -27649,0 -27650,0 -27651,0 -27652,0 -27653,0 -27654,0 -27655,0 -27656,0 -27657,0 -27658,0 -27659,0 -27660,0 -27661,0 -27662,0 -27663,0 -27664,0 -27665,0 -27666,0 -27667,0 -27668,0 -27669,0 -27670,0 -27671,0 -27672,0 -27673,0 -27674,0 -27675,0 -27676,0 -27677,0 -27678,0 -27679,0 -27680,0 -27681,0 -27682,0 -27683,0 -27684,0 -27685,0 -27686,0 -27687,0 -27688,0 -27689,0 -27690,0 -27691,0 -27692,0 -27693,0 -27694,0 -27695,0 -27696,0 -27697,0 -27698,0 -27699,0 -27700,0 -27701,0 -27702,0 -27703,0 -27704,0 -27705,0 -27706,0 -27707,0 -27708,0 -27709,0 -27710,0 -27711,0 -27712,0 -27713,0 -27714,0 -27715,0 -27716,0 -27717,0 -27718,0 -27719,0 -27720,0 -27721,0 -27722,0 -27723,0 -27724,0 -27725,0 -27726,0 -27727,0 -27728,0 -27729,0 -27730,0 -27731,0 -27732,0 -27733,0 -27734,0 -27735,0 -27736,0 -27737,0 -27738,0 -27739,0 -27740,0 -27741,0 -27742,0 -27743,0 -27744,0 -27745,0 -27746,0 -27747,0 -27748,0 -27749,0 -27750,0 -27751,0 -27752,0 -27753,0 -27754,0 -27755,0 -27756,0 -27757,0 -27758,0 -27759,0 -27760,0 -27761,0 -27762,0 -27763,0 -27764,0 -27765,0 -27766,0 -27767,0 -27768,0 -27769,0 -27770,0 -27771,0 -27772,0 -27773,0 -27774,0 -27775,0 -27776,0 -27777,0 -27778,0 -27779,0 -27780,0 -27781,0 -27782,0 -27783,0 -27784,0 -27785,0 -27786,0 -27787,0 -27788,0 -27789,0 -27790,0 -27791,0 -27792,0 -27793,0 -27794,0 -27795,0 -27796,0 -27797,0 -27798,0 -27799,0 -27800,0 -27801,0 -27802,0 -27803,0 -27804,0 -27805,0 -27806,0 -27807,0 -27808,0 -27809,0 -27810,0 -27811,0 -27812,0 -27813,0 -27814,0 -27815,0 -27816,0 -27817,0 -27818,0 -27819,0 -27820,0 -27821,0 -27822,0 -27823,0 -27824,0 -27825,0 -27826,0 -27827,0 -27828,0 -27829,0 -27830,0 -27831,0 -27832,0 -27833,0 -27834,0 -27835,0 -27836,0 -27837,0 -27838,0 -27839,0 -27840,0 -27841,0 -27842,0 -27843,0 -27844,0 -27845,0 -27846,0 -27847,0 -27848,0 -27849,0 -27850,0 -27851,0 -27852,0 -27853,0 -27854,0 -27855,0 -27856,0 -27857,0 -27858,0 -27859,0 -27860,0 -27861,0 -27862,0 -27863,0 -27864,0 -27865,0 -27866,0 -27867,0 -27868,0 -27869,0 -27870,0 -27871,0 -27872,0 -27873,0 -27874,0 -27875,0 -27876,0 -27877,0 -27878,0 -27879,0 -27880,0 -27881,0 -27882,0 -27883,0 -27884,0 -27885,0 -27886,0 -27887,0 -27888,0 -27889,0 -27890,0 -27891,0 -27892,0 -27893,0 -27894,0 -27895,0 -27896,0 -27897,0 -27898,0 -27899,0 -27900,0 -27901,0 -27902,0 -27903,0 -27904,0 -27905,0 -27906,0 -27907,0 -27908,0 -27909,0 -27910,0 -27911,0 -27912,0 -27913,0 -27914,0 -27915,0 -27916,0 -27917,0 -27918,0 -27919,0 -27920,0 -27921,0 -27922,0 -27923,0 -27924,0 -27925,0 -27926,0 -27927,0 -27928,0 -27929,0 -27930,0 -27931,0 -27932,0 -27933,0 -27934,0 -27935,0 -27936,0 -27937,0 -27938,0 -27939,0 -27940,0 -27941,0 -27942,0 -27943,0 -27944,0 -27945,0 -27946,0 -27947,0 -27948,0 -27949,0 -27950,0 -27951,0 -27952,0 -27953,0 -27954,0 -27955,0 -27956,0 -27957,0 -27958,0 -27959,0 -27960,0 -27961,0 -27962,0 -27963,0 -27964,0 -27965,0 -27966,0 -27967,0 -27968,0 -27969,0 -27970,0 -27971,0 -27972,0 -27973,0 -27974,0 -27975,0 -27976,0 -27977,0 -27978,0 -27979,0 -27980,0 -27981,0 -27982,0 -27983,0 -27984,0 -27985,0 -27986,0 -27987,0 -27988,0 -27989,0 -27990,0 -27991,0 -27992,0 -27993,0 -27994,0 -27995,0 -27996,0 -27997,0 -27998,0 -27999,0 -28000,0 +ImageId,Label +1,0 +2,0 +3,0 +4,0 +5,0 +6,0 +7,0 +8,0 +9,0 +10,0 +11,0 +12,0 +13,0 +14,0 +15,0 +16,0 +17,0 +18,0 +19,0 +20,0 +21,0 +22,0 +23,0 +24,0 +25,0 +26,0 +27,0 +28,0 +29,0 +30,0 +31,0 +32,0 +33,0 +34,0 +35,0 +36,0 +37,0 +38,0 +39,0 +40,0 +41,0 +42,0 +43,0 +44,0 +45,0 +46,0 +47,0 +48,0 +49,0 +50,0 +51,0 +52,0 +53,0 +54,0 +55,0 +56,0 +57,0 +58,0 +59,0 +60,0 +61,0 +62,0 +63,0 +64,0 +65,0 +66,0 +67,0 +68,0 +69,0 +70,0 +71,0 +72,0 +73,0 +74,0 +75,0 +76,0 +77,0 +78,0 +79,0 +80,0 +81,0 +82,0 +83,0 +84,0 +85,0 +86,0 +87,0 +88,0 +89,0 +90,0 +91,0 +92,0 +93,0 +94,0 +95,0 +96,0 +97,0 +98,0 +99,0 +100,0 +101,0 +102,0 +103,0 +104,0 +105,0 +106,0 +107,0 +108,0 +109,0 +110,0 +111,0 +112,0 +113,0 +114,0 +115,0 +116,0 +117,0 +118,0 +119,0 +120,0 +121,0 +122,0 +123,0 +124,0 +125,0 +126,0 +127,0 +128,0 +129,0 +130,0 +131,0 +132,0 +133,0 +134,0 +135,0 +136,0 +137,0 +138,0 +139,0 +140,0 +141,0 +142,0 +143,0 +144,0 +145,0 +146,0 +147,0 +148,0 +149,0 +150,0 +151,0 +152,0 +153,0 +154,0 +155,0 +156,0 +157,0 +158,0 +159,0 +160,0 +161,0 +162,0 +163,0 +164,0 +165,0 +166,0 +167,0 +168,0 +169,0 +170,0 +171,0 +172,0 +173,0 +174,0 +175,0 +176,0 +177,0 +178,0 +179,0 +180,0 +181,0 +182,0 +183,0 +184,0 +185,0 +186,0 +187,0 +188,0 +189,0 +190,0 +191,0 +192,0 +193,0 +194,0 +195,0 +196,0 +197,0 +198,0 +199,0 +200,0 +201,0 +202,0 +203,0 +204,0 +205,0 +206,0 +207,0 +208,0 +209,0 +210,0 +211,0 +212,0 +213,0 +214,0 +215,0 +216,0 +217,0 +218,0 +219,0 +220,0 +221,0 +222,0 +223,0 +224,0 +225,0 +226,0 +227,0 +228,0 +229,0 +230,0 +231,0 +232,0 +233,0 +234,0 +235,0 +236,0 +237,0 +238,0 +239,0 +240,0 +241,0 +242,0 +243,0 +244,0 +245,0 +246,0 +247,0 +248,0 +249,0 +250,0 +251,0 +252,0 +253,0 +254,0 +255,0 +256,0 +257,0 +258,0 +259,0 +260,0 +261,0 +262,0 +263,0 +264,0 +265,0 +266,0 +267,0 +268,0 +269,0 +270,0 +271,0 +272,0 +273,0 +274,0 +275,0 +276,0 +277,0 +278,0 +279,0 +280,0 +281,0 +282,0 +283,0 +284,0 +285,0 +286,0 +287,0 +288,0 +289,0 +290,0 +291,0 +292,0 +293,0 +294,0 +295,0 +296,0 +297,0 +298,0 +299,0 +300,0 +301,0 +302,0 +303,0 +304,0 +305,0 +306,0 +307,0 +308,0 +309,0 +310,0 +311,0 +312,0 +313,0 +314,0 +315,0 +316,0 +317,0 +318,0 +319,0 +320,0 +321,0 +322,0 +323,0 +324,0 +325,0 +326,0 +327,0 +328,0 +329,0 +330,0 +331,0 +332,0 +333,0 +334,0 +335,0 +336,0 +337,0 +338,0 +339,0 +340,0 +341,0 +342,0 +343,0 +344,0 +345,0 +346,0 +347,0 +348,0 +349,0 +350,0 +351,0 +352,0 +353,0 +354,0 +355,0 +356,0 +357,0 +358,0 +359,0 +360,0 +361,0 +362,0 +363,0 +364,0 +365,0 +366,0 +367,0 +368,0 +369,0 +370,0 +371,0 +372,0 +373,0 +374,0 +375,0 +376,0 +377,0 +378,0 +379,0 +380,0 +381,0 +382,0 +383,0 +384,0 +385,0 +386,0 +387,0 +388,0 +389,0 +390,0 +391,0 +392,0 +393,0 +394,0 +395,0 +396,0 +397,0 +398,0 +399,0 +400,0 +401,0 +402,0 +403,0 +404,0 +405,0 +406,0 +407,0 +408,0 +409,0 +410,0 +411,0 +412,0 +413,0 +414,0 +415,0 +416,0 +417,0 +418,0 +419,0 +420,0 +421,0 +422,0 +423,0 +424,0 +425,0 +426,0 +427,0 +428,0 +429,0 +430,0 +431,0 +432,0 +433,0 +434,0 +435,0 +436,0 +437,0 +438,0 +439,0 +440,0 +441,0 +442,0 +443,0 +444,0 +445,0 +446,0 +447,0 +448,0 +449,0 +450,0 +451,0 +452,0 +453,0 +454,0 +455,0 +456,0 +457,0 +458,0 +459,0 +460,0 +461,0 +462,0 +463,0 +464,0 +465,0 +466,0 +467,0 +468,0 +469,0 +470,0 +471,0 +472,0 +473,0 +474,0 +475,0 +476,0 +477,0 +478,0 +479,0 +480,0 +481,0 +482,0 +483,0 +484,0 +485,0 +486,0 +487,0 +488,0 +489,0 +490,0 +491,0 +492,0 +493,0 +494,0 +495,0 +496,0 +497,0 +498,0 +499,0 +500,0 +501,0 +502,0 +503,0 +504,0 +505,0 +506,0 +507,0 +508,0 +509,0 +510,0 +511,0 +512,0 +513,0 +514,0 +515,0 +516,0 +517,0 +518,0 +519,0 +520,0 +521,0 +522,0 +523,0 +524,0 +525,0 +526,0 +527,0 +528,0 +529,0 +530,0 +531,0 +532,0 +533,0 +534,0 +535,0 +536,0 +537,0 +538,0 +539,0 +540,0 +541,0 +542,0 +543,0 +544,0 +545,0 +546,0 +547,0 +548,0 +549,0 +550,0 +551,0 +552,0 +553,0 +554,0 +555,0 +556,0 +557,0 +558,0 +559,0 +560,0 +561,0 +562,0 +563,0 +564,0 +565,0 +566,0 +567,0 +568,0 +569,0 +570,0 +571,0 +572,0 +573,0 +574,0 +575,0 +576,0 +577,0 +578,0 +579,0 +580,0 +581,0 +582,0 +583,0 +584,0 +585,0 +586,0 +587,0 +588,0 +589,0 +590,0 +591,0 +592,0 +593,0 +594,0 +595,0 +596,0 +597,0 +598,0 +599,0 +600,0 +601,0 +602,0 +603,0 +604,0 +605,0 +606,0 +607,0 +608,0 +609,0 +610,0 +611,0 +612,0 +613,0 +614,0 +615,0 +616,0 +617,0 +618,0 +619,0 +620,0 +621,0 +622,0 +623,0 +624,0 +625,0 +626,0 +627,0 +628,0 +629,0 +630,0 +631,0 +632,0 +633,0 +634,0 +635,0 +636,0 +637,0 +638,0 +639,0 +640,0 +641,0 +642,0 +643,0 +644,0 +645,0 +646,0 +647,0 +648,0 +649,0 +650,0 +651,0 +652,0 +653,0 +654,0 +655,0 +656,0 +657,0 +658,0 +659,0 +660,0 +661,0 +662,0 +663,0 +664,0 +665,0 +666,0 +667,0 +668,0 +669,0 +670,0 +671,0 +672,0 +673,0 +674,0 +675,0 +676,0 +677,0 +678,0 +679,0 +680,0 +681,0 +682,0 +683,0 +684,0 +685,0 +686,0 +687,0 +688,0 +689,0 +690,0 +691,0 +692,0 +693,0 +694,0 +695,0 +696,0 +697,0 +698,0 +699,0 +700,0 +701,0 +702,0 +703,0 +704,0 +705,0 +706,0 +707,0 +708,0 +709,0 +710,0 +711,0 +712,0 +713,0 +714,0 +715,0 +716,0 +717,0 +718,0 +719,0 +720,0 +721,0 +722,0 +723,0 +724,0 +725,0 +726,0 +727,0 +728,0 +729,0 +730,0 +731,0 +732,0 +733,0 +734,0 +735,0 +736,0 +737,0 +738,0 +739,0 +740,0 +741,0 +742,0 +743,0 +744,0 +745,0 +746,0 +747,0 +748,0 +749,0 +750,0 +751,0 +752,0 +753,0 +754,0 +755,0 +756,0 +757,0 +758,0 +759,0 +760,0 +761,0 +762,0 +763,0 +764,0 +765,0 +766,0 +767,0 +768,0 +769,0 +770,0 +771,0 +772,0 +773,0 +774,0 +775,0 +776,0 +777,0 +778,0 +779,0 +780,0 +781,0 +782,0 +783,0 +784,0 +785,0 +786,0 +787,0 +788,0 +789,0 +790,0 +791,0 +792,0 +793,0 +794,0 +795,0 +796,0 +797,0 +798,0 +799,0 +800,0 +801,0 +802,0 +803,0 +804,0 +805,0 +806,0 +807,0 +808,0 +809,0 +810,0 +811,0 +812,0 +813,0 +814,0 +815,0 +816,0 +817,0 +818,0 +819,0 +820,0 +821,0 +822,0 +823,0 +824,0 +825,0 +826,0 +827,0 +828,0 +829,0 +830,0 +831,0 +832,0 +833,0 +834,0 +835,0 +836,0 +837,0 +838,0 +839,0 +840,0 +841,0 +842,0 +843,0 +844,0 +845,0 +846,0 +847,0 +848,0 +849,0 +850,0 +851,0 +852,0 +853,0 +854,0 +855,0 +856,0 +857,0 +858,0 +859,0 +860,0 +861,0 +862,0 +863,0 +864,0 +865,0 +866,0 +867,0 +868,0 +869,0 +870,0 +871,0 +872,0 +873,0 +874,0 +875,0 +876,0 +877,0 +878,0 +879,0 +880,0 +881,0 +882,0 +883,0 +884,0 +885,0 +886,0 +887,0 +888,0 +889,0 +890,0 +891,0 +892,0 +893,0 +894,0 +895,0 +896,0 +897,0 +898,0 +899,0 +900,0 +901,0 +902,0 +903,0 +904,0 +905,0 +906,0 +907,0 +908,0 +909,0 +910,0 +911,0 +912,0 +913,0 +914,0 +915,0 +916,0 +917,0 +918,0 +919,0 +920,0 +921,0 +922,0 +923,0 +924,0 +925,0 +926,0 +927,0 +928,0 +929,0 +930,0 +931,0 +932,0 +933,0 +934,0 +935,0 +936,0 +937,0 +938,0 +939,0 +940,0 +941,0 +942,0 +943,0 +944,0 +945,0 +946,0 +947,0 +948,0 +949,0 +950,0 +951,0 +952,0 +953,0 +954,0 +955,0 +956,0 +957,0 +958,0 +959,0 +960,0 +961,0 +962,0 +963,0 +964,0 +965,0 +966,0 +967,0 +968,0 +969,0 +970,0 +971,0 +972,0 +973,0 +974,0 +975,0 +976,0 +977,0 +978,0 +979,0 +980,0 +981,0 +982,0 +983,0 +984,0 +985,0 +986,0 +987,0 +988,0 +989,0 +990,0 +991,0 +992,0 +993,0 +994,0 +995,0 +996,0 +997,0 +998,0 +999,0 +1000,0 +1001,0 +1002,0 +1003,0 +1004,0 +1005,0 +1006,0 +1007,0 +1008,0 +1009,0 +1010,0 +1011,0 +1012,0 +1013,0 +1014,0 +1015,0 +1016,0 +1017,0 +1018,0 +1019,0 +1020,0 +1021,0 +1022,0 +1023,0 +1024,0 +1025,0 +1026,0 +1027,0 +1028,0 +1029,0 +1030,0 +1031,0 +1032,0 +1033,0 +1034,0 +1035,0 +1036,0 +1037,0 +1038,0 +1039,0 +1040,0 +1041,0 +1042,0 +1043,0 +1044,0 +1045,0 +1046,0 +1047,0 +1048,0 +1049,0 +1050,0 +1051,0 +1052,0 +1053,0 +1054,0 +1055,0 +1056,0 +1057,0 +1058,0 +1059,0 +1060,0 +1061,0 +1062,0 +1063,0 +1064,0 +1065,0 +1066,0 +1067,0 +1068,0 +1069,0 +1070,0 +1071,0 +1072,0 +1073,0 +1074,0 +1075,0 +1076,0 +1077,0 +1078,0 +1079,0 +1080,0 +1081,0 +1082,0 +1083,0 +1084,0 +1085,0 +1086,0 +1087,0 +1088,0 +1089,0 +1090,0 +1091,0 +1092,0 +1093,0 +1094,0 +1095,0 +1096,0 +1097,0 +1098,0 +1099,0 +1100,0 +1101,0 +1102,0 +1103,0 +1104,0 +1105,0 +1106,0 +1107,0 +1108,0 +1109,0 +1110,0 +1111,0 +1112,0 +1113,0 +1114,0 +1115,0 +1116,0 +1117,0 +1118,0 +1119,0 +1120,0 +1121,0 +1122,0 +1123,0 +1124,0 +1125,0 +1126,0 +1127,0 +1128,0 +1129,0 +1130,0 +1131,0 +1132,0 +1133,0 +1134,0 +1135,0 +1136,0 +1137,0 +1138,0 +1139,0 +1140,0 +1141,0 +1142,0 +1143,0 +1144,0 +1145,0 +1146,0 +1147,0 +1148,0 +1149,0 +1150,0 +1151,0 +1152,0 +1153,0 +1154,0 +1155,0 +1156,0 +1157,0 +1158,0 +1159,0 +1160,0 +1161,0 +1162,0 +1163,0 +1164,0 +1165,0 +1166,0 +1167,0 +1168,0 +1169,0 +1170,0 +1171,0 +1172,0 +1173,0 +1174,0 +1175,0 +1176,0 +1177,0 +1178,0 +1179,0 +1180,0 +1181,0 +1182,0 +1183,0 +1184,0 +1185,0 +1186,0 +1187,0 +1188,0 +1189,0 +1190,0 +1191,0 +1192,0 +1193,0 +1194,0 +1195,0 +1196,0 +1197,0 +1198,0 +1199,0 +1200,0 +1201,0 +1202,0 +1203,0 +1204,0 +1205,0 +1206,0 +1207,0 +1208,0 +1209,0 +1210,0 +1211,0 +1212,0 +1213,0 +1214,0 +1215,0 +1216,0 +1217,0 +1218,0 +1219,0 +1220,0 +1221,0 +1222,0 +1223,0 +1224,0 +1225,0 +1226,0 +1227,0 +1228,0 +1229,0 +1230,0 +1231,0 +1232,0 +1233,0 +1234,0 +1235,0 +1236,0 +1237,0 +1238,0 +1239,0 +1240,0 +1241,0 +1242,0 +1243,0 +1244,0 +1245,0 +1246,0 +1247,0 +1248,0 +1249,0 +1250,0 +1251,0 +1252,0 +1253,0 +1254,0 +1255,0 +1256,0 +1257,0 +1258,0 +1259,0 +1260,0 +1261,0 +1262,0 +1263,0 +1264,0 +1265,0 +1266,0 +1267,0 +1268,0 +1269,0 +1270,0 +1271,0 +1272,0 +1273,0 +1274,0 +1275,0 +1276,0 +1277,0 +1278,0 +1279,0 +1280,0 +1281,0 +1282,0 +1283,0 +1284,0 +1285,0 +1286,0 +1287,0 +1288,0 +1289,0 +1290,0 +1291,0 +1292,0 +1293,0 +1294,0 +1295,0 +1296,0 +1297,0 +1298,0 +1299,0 +1300,0 +1301,0 +1302,0 +1303,0 +1304,0 +1305,0 +1306,0 +1307,0 +1308,0 +1309,0 +1310,0 +1311,0 +1312,0 +1313,0 +1314,0 +1315,0 +1316,0 +1317,0 +1318,0 +1319,0 +1320,0 +1321,0 +1322,0 +1323,0 +1324,0 +1325,0 +1326,0 +1327,0 +1328,0 +1329,0 +1330,0 +1331,0 +1332,0 +1333,0 +1334,0 +1335,0 +1336,0 +1337,0 +1338,0 +1339,0 +1340,0 +1341,0 +1342,0 +1343,0 +1344,0 +1345,0 +1346,0 +1347,0 +1348,0 +1349,0 +1350,0 +1351,0 +1352,0 +1353,0 +1354,0 +1355,0 +1356,0 +1357,0 +1358,0 +1359,0 +1360,0 +1361,0 +1362,0 +1363,0 +1364,0 +1365,0 +1366,0 +1367,0 +1368,0 +1369,0 +1370,0 +1371,0 +1372,0 +1373,0 +1374,0 +1375,0 +1376,0 +1377,0 +1378,0 +1379,0 +1380,0 +1381,0 +1382,0 +1383,0 +1384,0 +1385,0 +1386,0 +1387,0 +1388,0 +1389,0 +1390,0 +1391,0 +1392,0 +1393,0 +1394,0 +1395,0 +1396,0 +1397,0 +1398,0 +1399,0 +1400,0 +1401,0 +1402,0 +1403,0 +1404,0 +1405,0 +1406,0 +1407,0 +1408,0 +1409,0 +1410,0 +1411,0 +1412,0 +1413,0 +1414,0 +1415,0 +1416,0 +1417,0 +1418,0 +1419,0 +1420,0 +1421,0 +1422,0 +1423,0 +1424,0 +1425,0 +1426,0 +1427,0 +1428,0 +1429,0 +1430,0 +1431,0 +1432,0 +1433,0 +1434,0 +1435,0 +1436,0 +1437,0 +1438,0 +1439,0 +1440,0 +1441,0 +1442,0 +1443,0 +1444,0 +1445,0 +1446,0 +1447,0 +1448,0 +1449,0 +1450,0 +1451,0 +1452,0 +1453,0 +1454,0 +1455,0 +1456,0 +1457,0 +1458,0 +1459,0 +1460,0 +1461,0 +1462,0 +1463,0 +1464,0 +1465,0 +1466,0 +1467,0 +1468,0 +1469,0 +1470,0 +1471,0 +1472,0 +1473,0 +1474,0 +1475,0 +1476,0 +1477,0 +1478,0 +1479,0 +1480,0 +1481,0 +1482,0 +1483,0 +1484,0 +1485,0 +1486,0 +1487,0 +1488,0 +1489,0 +1490,0 +1491,0 +1492,0 +1493,0 +1494,0 +1495,0 +1496,0 +1497,0 +1498,0 +1499,0 +1500,0 +1501,0 +1502,0 +1503,0 +1504,0 +1505,0 +1506,0 +1507,0 +1508,0 +1509,0 +1510,0 +1511,0 +1512,0 +1513,0 +1514,0 +1515,0 +1516,0 +1517,0 +1518,0 +1519,0 +1520,0 +1521,0 +1522,0 +1523,0 +1524,0 +1525,0 +1526,0 +1527,0 +1528,0 +1529,0 +1530,0 +1531,0 +1532,0 +1533,0 +1534,0 +1535,0 +1536,0 +1537,0 +1538,0 +1539,0 +1540,0 +1541,0 +1542,0 +1543,0 +1544,0 +1545,0 +1546,0 +1547,0 +1548,0 +1549,0 +1550,0 +1551,0 +1552,0 +1553,0 +1554,0 +1555,0 +1556,0 +1557,0 +1558,0 +1559,0 +1560,0 +1561,0 +1562,0 +1563,0 +1564,0 +1565,0 +1566,0 +1567,0 +1568,0 +1569,0 +1570,0 +1571,0 +1572,0 +1573,0 +1574,0 +1575,0 +1576,0 +1577,0 +1578,0 +1579,0 +1580,0 +1581,0 +1582,0 +1583,0 +1584,0 +1585,0 +1586,0 +1587,0 +1588,0 +1589,0 +1590,0 +1591,0 +1592,0 +1593,0 +1594,0 +1595,0 +1596,0 +1597,0 +1598,0 +1599,0 +1600,0 +1601,0 +1602,0 +1603,0 +1604,0 +1605,0 +1606,0 +1607,0 +1608,0 +1609,0 +1610,0 +1611,0 +1612,0 +1613,0 +1614,0 +1615,0 +1616,0 +1617,0 +1618,0 +1619,0 +1620,0 +1621,0 +1622,0 +1623,0 +1624,0 +1625,0 +1626,0 +1627,0 +1628,0 +1629,0 +1630,0 +1631,0 +1632,0 +1633,0 +1634,0 +1635,0 +1636,0 +1637,0 +1638,0 +1639,0 +1640,0 +1641,0 +1642,0 +1643,0 +1644,0 +1645,0 +1646,0 +1647,0 +1648,0 +1649,0 +1650,0 +1651,0 +1652,0 +1653,0 +1654,0 +1655,0 +1656,0 +1657,0 +1658,0 +1659,0 +1660,0 +1661,0 +1662,0 +1663,0 +1664,0 +1665,0 +1666,0 +1667,0 +1668,0 +1669,0 +1670,0 +1671,0 +1672,0 +1673,0 +1674,0 +1675,0 +1676,0 +1677,0 +1678,0 +1679,0 +1680,0 +1681,0 +1682,0 +1683,0 +1684,0 +1685,0 +1686,0 +1687,0 +1688,0 +1689,0 +1690,0 +1691,0 +1692,0 +1693,0 +1694,0 +1695,0 +1696,0 +1697,0 +1698,0 +1699,0 +1700,0 +1701,0 +1702,0 +1703,0 +1704,0 +1705,0 +1706,0 +1707,0 +1708,0 +1709,0 +1710,0 +1711,0 +1712,0 +1713,0 +1714,0 +1715,0 +1716,0 +1717,0 +1718,0 +1719,0 +1720,0 +1721,0 +1722,0 +1723,0 +1724,0 +1725,0 +1726,0 +1727,0 +1728,0 +1729,0 +1730,0 +1731,0 +1732,0 +1733,0 +1734,0 +1735,0 +1736,0 +1737,0 +1738,0 +1739,0 +1740,0 +1741,0 +1742,0 +1743,0 +1744,0 +1745,0 +1746,0 +1747,0 +1748,0 +1749,0 +1750,0 +1751,0 +1752,0 +1753,0 +1754,0 +1755,0 +1756,0 +1757,0 +1758,0 +1759,0 +1760,0 +1761,0 +1762,0 +1763,0 +1764,0 +1765,0 +1766,0 +1767,0 +1768,0 +1769,0 +1770,0 +1771,0 +1772,0 +1773,0 +1774,0 +1775,0 +1776,0 +1777,0 +1778,0 +1779,0 +1780,0 +1781,0 +1782,0 +1783,0 +1784,0 +1785,0 +1786,0 +1787,0 +1788,0 +1789,0 +1790,0 +1791,0 +1792,0 +1793,0 +1794,0 +1795,0 +1796,0 +1797,0 +1798,0 +1799,0 +1800,0 +1801,0 +1802,0 +1803,0 +1804,0 +1805,0 +1806,0 +1807,0 +1808,0 +1809,0 +1810,0 +1811,0 +1812,0 +1813,0 +1814,0 +1815,0 +1816,0 +1817,0 +1818,0 +1819,0 +1820,0 +1821,0 +1822,0 +1823,0 +1824,0 +1825,0 +1826,0 +1827,0 +1828,0 +1829,0 +1830,0 +1831,0 +1832,0 +1833,0 +1834,0 +1835,0 +1836,0 +1837,0 +1838,0 +1839,0 +1840,0 +1841,0 +1842,0 +1843,0 +1844,0 +1845,0 +1846,0 +1847,0 +1848,0 +1849,0 +1850,0 +1851,0 +1852,0 +1853,0 +1854,0 +1855,0 +1856,0 +1857,0 +1858,0 +1859,0 +1860,0 +1861,0 +1862,0 +1863,0 +1864,0 +1865,0 +1866,0 +1867,0 +1868,0 +1869,0 +1870,0 +1871,0 +1872,0 +1873,0 +1874,0 +1875,0 +1876,0 +1877,0 +1878,0 +1879,0 +1880,0 +1881,0 +1882,0 +1883,0 +1884,0 +1885,0 +1886,0 +1887,0 +1888,0 +1889,0 +1890,0 +1891,0 +1892,0 +1893,0 +1894,0 +1895,0 +1896,0 +1897,0 +1898,0 +1899,0 +1900,0 +1901,0 +1902,0 +1903,0 +1904,0 +1905,0 +1906,0 +1907,0 +1908,0 +1909,0 +1910,0 +1911,0 +1912,0 +1913,0 +1914,0 +1915,0 +1916,0 +1917,0 +1918,0 +1919,0 +1920,0 +1921,0 +1922,0 +1923,0 +1924,0 +1925,0 +1926,0 +1927,0 +1928,0 +1929,0 +1930,0 +1931,0 +1932,0 +1933,0 +1934,0 +1935,0 +1936,0 +1937,0 +1938,0 +1939,0 +1940,0 +1941,0 +1942,0 +1943,0 +1944,0 +1945,0 +1946,0 +1947,0 +1948,0 +1949,0 +1950,0 +1951,0 +1952,0 +1953,0 +1954,0 +1955,0 +1956,0 +1957,0 +1958,0 +1959,0 +1960,0 +1961,0 +1962,0 +1963,0 +1964,0 +1965,0 +1966,0 +1967,0 +1968,0 +1969,0 +1970,0 +1971,0 +1972,0 +1973,0 +1974,0 +1975,0 +1976,0 +1977,0 +1978,0 +1979,0 +1980,0 +1981,0 +1982,0 +1983,0 +1984,0 +1985,0 +1986,0 +1987,0 +1988,0 +1989,0 +1990,0 +1991,0 +1992,0 +1993,0 +1994,0 +1995,0 +1996,0 +1997,0 +1998,0 +1999,0 +2000,0 +2001,0 +2002,0 +2003,0 +2004,0 +2005,0 +2006,0 +2007,0 +2008,0 +2009,0 +2010,0 +2011,0 +2012,0 +2013,0 +2014,0 +2015,0 +2016,0 +2017,0 +2018,0 +2019,0 +2020,0 +2021,0 +2022,0 +2023,0 +2024,0 +2025,0 +2026,0 +2027,0 +2028,0 +2029,0 +2030,0 +2031,0 +2032,0 +2033,0 +2034,0 +2035,0 +2036,0 +2037,0 +2038,0 +2039,0 +2040,0 +2041,0 +2042,0 +2043,0 +2044,0 +2045,0 +2046,0 +2047,0 +2048,0 +2049,0 +2050,0 +2051,0 +2052,0 +2053,0 +2054,0 +2055,0 +2056,0 +2057,0 +2058,0 +2059,0 +2060,0 +2061,0 +2062,0 +2063,0 +2064,0 +2065,0 +2066,0 +2067,0 +2068,0 +2069,0 +2070,0 +2071,0 +2072,0 +2073,0 +2074,0 +2075,0 +2076,0 +2077,0 +2078,0 +2079,0 +2080,0 +2081,0 +2082,0 +2083,0 +2084,0 +2085,0 +2086,0 +2087,0 +2088,0 +2089,0 +2090,0 +2091,0 +2092,0 +2093,0 +2094,0 +2095,0 +2096,0 +2097,0 +2098,0 +2099,0 +2100,0 +2101,0 +2102,0 +2103,0 +2104,0 +2105,0 +2106,0 +2107,0 +2108,0 +2109,0 +2110,0 +2111,0 +2112,0 +2113,0 +2114,0 +2115,0 +2116,0 +2117,0 +2118,0 +2119,0 +2120,0 +2121,0 +2122,0 +2123,0 +2124,0 +2125,0 +2126,0 +2127,0 +2128,0 +2129,0 +2130,0 +2131,0 +2132,0 +2133,0 +2134,0 +2135,0 +2136,0 +2137,0 +2138,0 +2139,0 +2140,0 +2141,0 +2142,0 +2143,0 +2144,0 +2145,0 +2146,0 +2147,0 +2148,0 +2149,0 +2150,0 +2151,0 +2152,0 +2153,0 +2154,0 +2155,0 +2156,0 +2157,0 +2158,0 +2159,0 +2160,0 +2161,0 +2162,0 +2163,0 +2164,0 +2165,0 +2166,0 +2167,0 +2168,0 +2169,0 +2170,0 +2171,0 +2172,0 +2173,0 +2174,0 +2175,0 +2176,0 +2177,0 +2178,0 +2179,0 +2180,0 +2181,0 +2182,0 +2183,0 +2184,0 +2185,0 +2186,0 +2187,0 +2188,0 +2189,0 +2190,0 +2191,0 +2192,0 +2193,0 +2194,0 +2195,0 +2196,0 +2197,0 +2198,0 +2199,0 +2200,0 +2201,0 +2202,0 +2203,0 +2204,0 +2205,0 +2206,0 +2207,0 +2208,0 +2209,0 +2210,0 +2211,0 +2212,0 +2213,0 +2214,0 +2215,0 +2216,0 +2217,0 +2218,0 +2219,0 +2220,0 +2221,0 +2222,0 +2223,0 +2224,0 +2225,0 +2226,0 +2227,0 +2228,0 +2229,0 +2230,0 +2231,0 +2232,0 +2233,0 +2234,0 +2235,0 +2236,0 +2237,0 +2238,0 +2239,0 +2240,0 +2241,0 +2242,0 +2243,0 +2244,0 +2245,0 +2246,0 +2247,0 +2248,0 +2249,0 +2250,0 +2251,0 +2252,0 +2253,0 +2254,0 +2255,0 +2256,0 +2257,0 +2258,0 +2259,0 +2260,0 +2261,0 +2262,0 +2263,0 +2264,0 +2265,0 +2266,0 +2267,0 +2268,0 +2269,0 +2270,0 +2271,0 +2272,0 +2273,0 +2274,0 +2275,0 +2276,0 +2277,0 +2278,0 +2279,0 +2280,0 +2281,0 +2282,0 +2283,0 +2284,0 +2285,0 +2286,0 +2287,0 +2288,0 +2289,0 +2290,0 +2291,0 +2292,0 +2293,0 +2294,0 +2295,0 +2296,0 +2297,0 +2298,0 +2299,0 +2300,0 +2301,0 +2302,0 +2303,0 +2304,0 +2305,0 +2306,0 +2307,0 +2308,0 +2309,0 +2310,0 +2311,0 +2312,0 +2313,0 +2314,0 +2315,0 +2316,0 +2317,0 +2318,0 +2319,0 +2320,0 +2321,0 +2322,0 +2323,0 +2324,0 +2325,0 +2326,0 +2327,0 +2328,0 +2329,0 +2330,0 +2331,0 +2332,0 +2333,0 +2334,0 +2335,0 +2336,0 +2337,0 +2338,0 +2339,0 +2340,0 +2341,0 +2342,0 +2343,0 +2344,0 +2345,0 +2346,0 +2347,0 +2348,0 +2349,0 +2350,0 +2351,0 +2352,0 +2353,0 +2354,0 +2355,0 +2356,0 +2357,0 +2358,0 +2359,0 +2360,0 +2361,0 +2362,0 +2363,0 +2364,0 +2365,0 +2366,0 +2367,0 +2368,0 +2369,0 +2370,0 +2371,0 +2372,0 +2373,0 +2374,0 +2375,0 +2376,0 +2377,0 +2378,0 +2379,0 +2380,0 +2381,0 +2382,0 +2383,0 +2384,0 +2385,0 +2386,0 +2387,0 +2388,0 +2389,0 +2390,0 +2391,0 +2392,0 +2393,0 +2394,0 +2395,0 +2396,0 +2397,0 +2398,0 +2399,0 +2400,0 +2401,0 +2402,0 +2403,0 +2404,0 +2405,0 +2406,0 +2407,0 +2408,0 +2409,0 +2410,0 +2411,0 +2412,0 +2413,0 +2414,0 +2415,0 +2416,0 +2417,0 +2418,0 +2419,0 +2420,0 +2421,0 +2422,0 +2423,0 +2424,0 +2425,0 +2426,0 +2427,0 +2428,0 +2429,0 +2430,0 +2431,0 +2432,0 +2433,0 +2434,0 +2435,0 +2436,0 +2437,0 +2438,0 +2439,0 +2440,0 +2441,0 +2442,0 +2443,0 +2444,0 +2445,0 +2446,0 +2447,0 +2448,0 +2449,0 +2450,0 +2451,0 +2452,0 +2453,0 +2454,0 +2455,0 +2456,0 +2457,0 +2458,0 +2459,0 +2460,0 +2461,0 +2462,0 +2463,0 +2464,0 +2465,0 +2466,0 +2467,0 +2468,0 +2469,0 +2470,0 +2471,0 +2472,0 +2473,0 +2474,0 +2475,0 +2476,0 +2477,0 +2478,0 +2479,0 +2480,0 +2481,0 +2482,0 +2483,0 +2484,0 +2485,0 +2486,0 +2487,0 +2488,0 +2489,0 +2490,0 +2491,0 +2492,0 +2493,0 +2494,0 +2495,0 +2496,0 +2497,0 +2498,0 +2499,0 +2500,0 +2501,0 +2502,0 +2503,0 +2504,0 +2505,0 +2506,0 +2507,0 +2508,0 +2509,0 +2510,0 +2511,0 +2512,0 +2513,0 +2514,0 +2515,0 +2516,0 +2517,0 +2518,0 +2519,0 +2520,0 +2521,0 +2522,0 +2523,0 +2524,0 +2525,0 +2526,0 +2527,0 +2528,0 +2529,0 +2530,0 +2531,0 +2532,0 +2533,0 +2534,0 +2535,0 +2536,0 +2537,0 +2538,0 +2539,0 +2540,0 +2541,0 +2542,0 +2543,0 +2544,0 +2545,0 +2546,0 +2547,0 +2548,0 +2549,0 +2550,0 +2551,0 +2552,0 +2553,0 +2554,0 +2555,0 +2556,0 +2557,0 +2558,0 +2559,0 +2560,0 +2561,0 +2562,0 +2563,0 +2564,0 +2565,0 +2566,0 +2567,0 +2568,0 +2569,0 +2570,0 +2571,0 +2572,0 +2573,0 +2574,0 +2575,0 +2576,0 +2577,0 +2578,0 +2579,0 +2580,0 +2581,0 +2582,0 +2583,0 +2584,0 +2585,0 +2586,0 +2587,0 +2588,0 +2589,0 +2590,0 +2591,0 +2592,0 +2593,0 +2594,0 +2595,0 +2596,0 +2597,0 +2598,0 +2599,0 +2600,0 +2601,0 +2602,0 +2603,0 +2604,0 +2605,0 +2606,0 +2607,0 +2608,0 +2609,0 +2610,0 +2611,0 +2612,0 +2613,0 +2614,0 +2615,0 +2616,0 +2617,0 +2618,0 +2619,0 +2620,0 +2621,0 +2622,0 +2623,0 +2624,0 +2625,0 +2626,0 +2627,0 +2628,0 +2629,0 +2630,0 +2631,0 +2632,0 +2633,0 +2634,0 +2635,0 +2636,0 +2637,0 +2638,0 +2639,0 +2640,0 +2641,0 +2642,0 +2643,0 +2644,0 +2645,0 +2646,0 +2647,0 +2648,0 +2649,0 +2650,0 +2651,0 +2652,0 +2653,0 +2654,0 +2655,0 +2656,0 +2657,0 +2658,0 +2659,0 +2660,0 +2661,0 +2662,0 +2663,0 +2664,0 +2665,0 +2666,0 +2667,0 +2668,0 +2669,0 +2670,0 +2671,0 +2672,0 +2673,0 +2674,0 +2675,0 +2676,0 +2677,0 +2678,0 +2679,0 +2680,0 +2681,0 +2682,0 +2683,0 +2684,0 +2685,0 +2686,0 +2687,0 +2688,0 +2689,0 +2690,0 +2691,0 +2692,0 +2693,0 +2694,0 +2695,0 +2696,0 +2697,0 +2698,0 +2699,0 +2700,0 +2701,0 +2702,0 +2703,0 +2704,0 +2705,0 +2706,0 +2707,0 +2708,0 +2709,0 +2710,0 +2711,0 +2712,0 +2713,0 +2714,0 +2715,0 +2716,0 +2717,0 +2718,0 +2719,0 +2720,0 +2721,0 +2722,0 +2723,0 +2724,0 +2725,0 +2726,0 +2727,0 +2728,0 +2729,0 +2730,0 +2731,0 +2732,0 +2733,0 +2734,0 +2735,0 +2736,0 +2737,0 +2738,0 +2739,0 +2740,0 +2741,0 +2742,0 +2743,0 +2744,0 +2745,0 +2746,0 +2747,0 +2748,0 +2749,0 +2750,0 +2751,0 +2752,0 +2753,0 +2754,0 +2755,0 +2756,0 +2757,0 +2758,0 +2759,0 +2760,0 +2761,0 +2762,0 +2763,0 +2764,0 +2765,0 +2766,0 +2767,0 +2768,0 +2769,0 +2770,0 +2771,0 +2772,0 +2773,0 +2774,0 +2775,0 +2776,0 +2777,0 +2778,0 +2779,0 +2780,0 +2781,0 +2782,0 +2783,0 +2784,0 +2785,0 +2786,0 +2787,0 +2788,0 +2789,0 +2790,0 +2791,0 +2792,0 +2793,0 +2794,0 +2795,0 +2796,0 +2797,0 +2798,0 +2799,0 +2800,0 +2801,0 +2802,0 +2803,0 +2804,0 +2805,0 +2806,0 +2807,0 +2808,0 +2809,0 +2810,0 +2811,0 +2812,0 +2813,0 +2814,0 +2815,0 +2816,0 +2817,0 +2818,0 +2819,0 +2820,0 +2821,0 +2822,0 +2823,0 +2824,0 +2825,0 +2826,0 +2827,0 +2828,0 +2829,0 +2830,0 +2831,0 +2832,0 +2833,0 +2834,0 +2835,0 +2836,0 +2837,0 +2838,0 +2839,0 +2840,0 +2841,0 +2842,0 +2843,0 +2844,0 +2845,0 +2846,0 +2847,0 +2848,0 +2849,0 +2850,0 +2851,0 +2852,0 +2853,0 +2854,0 +2855,0 +2856,0 +2857,0 +2858,0 +2859,0 +2860,0 +2861,0 +2862,0 +2863,0 +2864,0 +2865,0 +2866,0 +2867,0 +2868,0 +2869,0 +2870,0 +2871,0 +2872,0 +2873,0 +2874,0 +2875,0 +2876,0 +2877,0 +2878,0 +2879,0 +2880,0 +2881,0 +2882,0 +2883,0 +2884,0 +2885,0 +2886,0 +2887,0 +2888,0 +2889,0 +2890,0 +2891,0 +2892,0 +2893,0 +2894,0 +2895,0 +2896,0 +2897,0 +2898,0 +2899,0 +2900,0 +2901,0 +2902,0 +2903,0 +2904,0 +2905,0 +2906,0 +2907,0 +2908,0 +2909,0 +2910,0 +2911,0 +2912,0 +2913,0 +2914,0 +2915,0 +2916,0 +2917,0 +2918,0 +2919,0 +2920,0 +2921,0 +2922,0 +2923,0 +2924,0 +2925,0 +2926,0 +2927,0 +2928,0 +2929,0 +2930,0 +2931,0 +2932,0 +2933,0 +2934,0 +2935,0 +2936,0 +2937,0 +2938,0 +2939,0 +2940,0 +2941,0 +2942,0 +2943,0 +2944,0 +2945,0 +2946,0 +2947,0 +2948,0 +2949,0 +2950,0 +2951,0 +2952,0 +2953,0 +2954,0 +2955,0 +2956,0 +2957,0 +2958,0 +2959,0 +2960,0 +2961,0 +2962,0 +2963,0 +2964,0 +2965,0 +2966,0 +2967,0 +2968,0 +2969,0 +2970,0 +2971,0 +2972,0 +2973,0 +2974,0 +2975,0 +2976,0 +2977,0 +2978,0 +2979,0 +2980,0 +2981,0 +2982,0 +2983,0 +2984,0 +2985,0 +2986,0 +2987,0 +2988,0 +2989,0 +2990,0 +2991,0 +2992,0 +2993,0 +2994,0 +2995,0 +2996,0 +2997,0 +2998,0 +2999,0 +3000,0 +3001,0 +3002,0 +3003,0 +3004,0 +3005,0 +3006,0 +3007,0 +3008,0 +3009,0 +3010,0 +3011,0 +3012,0 +3013,0 +3014,0 +3015,0 +3016,0 +3017,0 +3018,0 +3019,0 +3020,0 +3021,0 +3022,0 +3023,0 +3024,0 +3025,0 +3026,0 +3027,0 +3028,0 +3029,0 +3030,0 +3031,0 +3032,0 +3033,0 +3034,0 +3035,0 +3036,0 +3037,0 +3038,0 +3039,0 +3040,0 +3041,0 +3042,0 +3043,0 +3044,0 +3045,0 +3046,0 +3047,0 +3048,0 +3049,0 +3050,0 +3051,0 +3052,0 +3053,0 +3054,0 +3055,0 +3056,0 +3057,0 +3058,0 +3059,0 +3060,0 +3061,0 +3062,0 +3063,0 +3064,0 +3065,0 +3066,0 +3067,0 +3068,0 +3069,0 +3070,0 +3071,0 +3072,0 +3073,0 +3074,0 +3075,0 +3076,0 +3077,0 +3078,0 +3079,0 +3080,0 +3081,0 +3082,0 +3083,0 +3084,0 +3085,0 +3086,0 +3087,0 +3088,0 +3089,0 +3090,0 +3091,0 +3092,0 +3093,0 +3094,0 +3095,0 +3096,0 +3097,0 +3098,0 +3099,0 +3100,0 +3101,0 +3102,0 +3103,0 +3104,0 +3105,0 +3106,0 +3107,0 +3108,0 +3109,0 +3110,0 +3111,0 +3112,0 +3113,0 +3114,0 +3115,0 +3116,0 +3117,0 +3118,0 +3119,0 +3120,0 +3121,0 +3122,0 +3123,0 +3124,0 +3125,0 +3126,0 +3127,0 +3128,0 +3129,0 +3130,0 +3131,0 +3132,0 +3133,0 +3134,0 +3135,0 +3136,0 +3137,0 +3138,0 +3139,0 +3140,0 +3141,0 +3142,0 +3143,0 +3144,0 +3145,0 +3146,0 +3147,0 +3148,0 +3149,0 +3150,0 +3151,0 +3152,0 +3153,0 +3154,0 +3155,0 +3156,0 +3157,0 +3158,0 +3159,0 +3160,0 +3161,0 +3162,0 +3163,0 +3164,0 +3165,0 +3166,0 +3167,0 +3168,0 +3169,0 +3170,0 +3171,0 +3172,0 +3173,0 +3174,0 +3175,0 +3176,0 +3177,0 +3178,0 +3179,0 +3180,0 +3181,0 +3182,0 +3183,0 +3184,0 +3185,0 +3186,0 +3187,0 +3188,0 +3189,0 +3190,0 +3191,0 +3192,0 +3193,0 +3194,0 +3195,0 +3196,0 +3197,0 +3198,0 +3199,0 +3200,0 +3201,0 +3202,0 +3203,0 +3204,0 +3205,0 +3206,0 +3207,0 +3208,0 +3209,0 +3210,0 +3211,0 +3212,0 +3213,0 +3214,0 +3215,0 +3216,0 +3217,0 +3218,0 +3219,0 +3220,0 +3221,0 +3222,0 +3223,0 +3224,0 +3225,0 +3226,0 +3227,0 +3228,0 +3229,0 +3230,0 +3231,0 +3232,0 +3233,0 +3234,0 +3235,0 +3236,0 +3237,0 +3238,0 +3239,0 +3240,0 +3241,0 +3242,0 +3243,0 +3244,0 +3245,0 +3246,0 +3247,0 +3248,0 +3249,0 +3250,0 +3251,0 +3252,0 +3253,0 +3254,0 +3255,0 +3256,0 +3257,0 +3258,0 +3259,0 +3260,0 +3261,0 +3262,0 +3263,0 +3264,0 +3265,0 +3266,0 +3267,0 +3268,0 +3269,0 +3270,0 +3271,0 +3272,0 +3273,0 +3274,0 +3275,0 +3276,0 +3277,0 +3278,0 +3279,0 +3280,0 +3281,0 +3282,0 +3283,0 +3284,0 +3285,0 +3286,0 +3287,0 +3288,0 +3289,0 +3290,0 +3291,0 +3292,0 +3293,0 +3294,0 +3295,0 +3296,0 +3297,0 +3298,0 +3299,0 +3300,0 +3301,0 +3302,0 +3303,0 +3304,0 +3305,0 +3306,0 +3307,0 +3308,0 +3309,0 +3310,0 +3311,0 +3312,0 +3313,0 +3314,0 +3315,0 +3316,0 +3317,0 +3318,0 +3319,0 +3320,0 +3321,0 +3322,0 +3323,0 +3324,0 +3325,0 +3326,0 +3327,0 +3328,0 +3329,0 +3330,0 +3331,0 +3332,0 +3333,0 +3334,0 +3335,0 +3336,0 +3337,0 +3338,0 +3339,0 +3340,0 +3341,0 +3342,0 +3343,0 +3344,0 +3345,0 +3346,0 +3347,0 +3348,0 +3349,0 +3350,0 +3351,0 +3352,0 +3353,0 +3354,0 +3355,0 +3356,0 +3357,0 +3358,0 +3359,0 +3360,0 +3361,0 +3362,0 +3363,0 +3364,0 +3365,0 +3366,0 +3367,0 +3368,0 +3369,0 +3370,0 +3371,0 +3372,0 +3373,0 +3374,0 +3375,0 +3376,0 +3377,0 +3378,0 +3379,0 +3380,0 +3381,0 +3382,0 +3383,0 +3384,0 +3385,0 +3386,0 +3387,0 +3388,0 +3389,0 +3390,0 +3391,0 +3392,0 +3393,0 +3394,0 +3395,0 +3396,0 +3397,0 +3398,0 +3399,0 +3400,0 +3401,0 +3402,0 +3403,0 +3404,0 +3405,0 +3406,0 +3407,0 +3408,0 +3409,0 +3410,0 +3411,0 +3412,0 +3413,0 +3414,0 +3415,0 +3416,0 +3417,0 +3418,0 +3419,0 +3420,0 +3421,0 +3422,0 +3423,0 +3424,0 +3425,0 +3426,0 +3427,0 +3428,0 +3429,0 +3430,0 +3431,0 +3432,0 +3433,0 +3434,0 +3435,0 +3436,0 +3437,0 +3438,0 +3439,0 +3440,0 +3441,0 +3442,0 +3443,0 +3444,0 +3445,0 +3446,0 +3447,0 +3448,0 +3449,0 +3450,0 +3451,0 +3452,0 +3453,0 +3454,0 +3455,0 +3456,0 +3457,0 +3458,0 +3459,0 +3460,0 +3461,0 +3462,0 +3463,0 +3464,0 +3465,0 +3466,0 +3467,0 +3468,0 +3469,0 +3470,0 +3471,0 +3472,0 +3473,0 +3474,0 +3475,0 +3476,0 +3477,0 +3478,0 +3479,0 +3480,0 +3481,0 +3482,0 +3483,0 +3484,0 +3485,0 +3486,0 +3487,0 +3488,0 +3489,0 +3490,0 +3491,0 +3492,0 +3493,0 +3494,0 +3495,0 +3496,0 +3497,0 +3498,0 +3499,0 +3500,0 +3501,0 +3502,0 +3503,0 +3504,0 +3505,0 +3506,0 +3507,0 +3508,0 +3509,0 +3510,0 +3511,0 +3512,0 +3513,0 +3514,0 +3515,0 +3516,0 +3517,0 +3518,0 +3519,0 +3520,0 +3521,0 +3522,0 +3523,0 +3524,0 +3525,0 +3526,0 +3527,0 +3528,0 +3529,0 +3530,0 +3531,0 +3532,0 +3533,0 +3534,0 +3535,0 +3536,0 +3537,0 +3538,0 +3539,0 +3540,0 +3541,0 +3542,0 +3543,0 +3544,0 +3545,0 +3546,0 +3547,0 +3548,0 +3549,0 +3550,0 +3551,0 +3552,0 +3553,0 +3554,0 +3555,0 +3556,0 +3557,0 +3558,0 +3559,0 +3560,0 +3561,0 +3562,0 +3563,0 +3564,0 +3565,0 +3566,0 +3567,0 +3568,0 +3569,0 +3570,0 +3571,0 +3572,0 +3573,0 +3574,0 +3575,0 +3576,0 +3577,0 +3578,0 +3579,0 +3580,0 +3581,0 +3582,0 +3583,0 +3584,0 +3585,0 +3586,0 +3587,0 +3588,0 +3589,0 +3590,0 +3591,0 +3592,0 +3593,0 +3594,0 +3595,0 +3596,0 +3597,0 +3598,0 +3599,0 +3600,0 +3601,0 +3602,0 +3603,0 +3604,0 +3605,0 +3606,0 +3607,0 +3608,0 +3609,0 +3610,0 +3611,0 +3612,0 +3613,0 +3614,0 +3615,0 +3616,0 +3617,0 +3618,0 +3619,0 +3620,0 +3621,0 +3622,0 +3623,0 +3624,0 +3625,0 +3626,0 +3627,0 +3628,0 +3629,0 +3630,0 +3631,0 +3632,0 +3633,0 +3634,0 +3635,0 +3636,0 +3637,0 +3638,0 +3639,0 +3640,0 +3641,0 +3642,0 +3643,0 +3644,0 +3645,0 +3646,0 +3647,0 +3648,0 +3649,0 +3650,0 +3651,0 +3652,0 +3653,0 +3654,0 +3655,0 +3656,0 +3657,0 +3658,0 +3659,0 +3660,0 +3661,0 +3662,0 +3663,0 +3664,0 +3665,0 +3666,0 +3667,0 +3668,0 +3669,0 +3670,0 +3671,0 +3672,0 +3673,0 +3674,0 +3675,0 +3676,0 +3677,0 +3678,0 +3679,0 +3680,0 +3681,0 +3682,0 +3683,0 +3684,0 +3685,0 +3686,0 +3687,0 +3688,0 +3689,0 +3690,0 +3691,0 +3692,0 +3693,0 +3694,0 +3695,0 +3696,0 +3697,0 +3698,0 +3699,0 +3700,0 +3701,0 +3702,0 +3703,0 +3704,0 +3705,0 +3706,0 +3707,0 +3708,0 +3709,0 +3710,0 +3711,0 +3712,0 +3713,0 +3714,0 +3715,0 +3716,0 +3717,0 +3718,0 +3719,0 +3720,0 +3721,0 +3722,0 +3723,0 +3724,0 +3725,0 +3726,0 +3727,0 +3728,0 +3729,0 +3730,0 +3731,0 +3732,0 +3733,0 +3734,0 +3735,0 +3736,0 +3737,0 +3738,0 +3739,0 +3740,0 +3741,0 +3742,0 +3743,0 +3744,0 +3745,0 +3746,0 +3747,0 +3748,0 +3749,0 +3750,0 +3751,0 +3752,0 +3753,0 +3754,0 +3755,0 +3756,0 +3757,0 +3758,0 +3759,0 +3760,0 +3761,0 +3762,0 +3763,0 +3764,0 +3765,0 +3766,0 +3767,0 +3768,0 +3769,0 +3770,0 +3771,0 +3772,0 +3773,0 +3774,0 +3775,0 +3776,0 +3777,0 +3778,0 +3779,0 +3780,0 +3781,0 +3782,0 +3783,0 +3784,0 +3785,0 +3786,0 +3787,0 +3788,0 +3789,0 +3790,0 +3791,0 +3792,0 +3793,0 +3794,0 +3795,0 +3796,0 +3797,0 +3798,0 +3799,0 +3800,0 +3801,0 +3802,0 +3803,0 +3804,0 +3805,0 +3806,0 +3807,0 +3808,0 +3809,0 +3810,0 +3811,0 +3812,0 +3813,0 +3814,0 +3815,0 +3816,0 +3817,0 +3818,0 +3819,0 +3820,0 +3821,0 +3822,0 +3823,0 +3824,0 +3825,0 +3826,0 +3827,0 +3828,0 +3829,0 +3830,0 +3831,0 +3832,0 +3833,0 +3834,0 +3835,0 +3836,0 +3837,0 +3838,0 +3839,0 +3840,0 +3841,0 +3842,0 +3843,0 +3844,0 +3845,0 +3846,0 +3847,0 +3848,0 +3849,0 +3850,0 +3851,0 +3852,0 +3853,0 +3854,0 +3855,0 +3856,0 +3857,0 +3858,0 +3859,0 +3860,0 +3861,0 +3862,0 +3863,0 +3864,0 +3865,0 +3866,0 +3867,0 +3868,0 +3869,0 +3870,0 +3871,0 +3872,0 +3873,0 +3874,0 +3875,0 +3876,0 +3877,0 +3878,0 +3879,0 +3880,0 +3881,0 +3882,0 +3883,0 +3884,0 +3885,0 +3886,0 +3887,0 +3888,0 +3889,0 +3890,0 +3891,0 +3892,0 +3893,0 +3894,0 +3895,0 +3896,0 +3897,0 +3898,0 +3899,0 +3900,0 +3901,0 +3902,0 +3903,0 +3904,0 +3905,0 +3906,0 +3907,0 +3908,0 +3909,0 +3910,0 +3911,0 +3912,0 +3913,0 +3914,0 +3915,0 +3916,0 +3917,0 +3918,0 +3919,0 +3920,0 +3921,0 +3922,0 +3923,0 +3924,0 +3925,0 +3926,0 +3927,0 +3928,0 +3929,0 +3930,0 +3931,0 +3932,0 +3933,0 +3934,0 +3935,0 +3936,0 +3937,0 +3938,0 +3939,0 +3940,0 +3941,0 +3942,0 +3943,0 +3944,0 +3945,0 +3946,0 +3947,0 +3948,0 +3949,0 +3950,0 +3951,0 +3952,0 +3953,0 +3954,0 +3955,0 +3956,0 +3957,0 +3958,0 +3959,0 +3960,0 +3961,0 +3962,0 +3963,0 +3964,0 +3965,0 +3966,0 +3967,0 +3968,0 +3969,0 +3970,0 +3971,0 +3972,0 +3973,0 +3974,0 +3975,0 +3976,0 +3977,0 +3978,0 +3979,0 +3980,0 +3981,0 +3982,0 +3983,0 +3984,0 +3985,0 +3986,0 +3987,0 +3988,0 +3989,0 +3990,0 +3991,0 +3992,0 +3993,0 +3994,0 +3995,0 +3996,0 +3997,0 +3998,0 +3999,0 +4000,0 +4001,0 +4002,0 +4003,0 +4004,0 +4005,0 +4006,0 +4007,0 +4008,0 +4009,0 +4010,0 +4011,0 +4012,0 +4013,0 +4014,0 +4015,0 +4016,0 +4017,0 +4018,0 +4019,0 +4020,0 +4021,0 +4022,0 +4023,0 +4024,0 +4025,0 +4026,0 +4027,0 +4028,0 +4029,0 +4030,0 +4031,0 +4032,0 +4033,0 +4034,0 +4035,0 +4036,0 +4037,0 +4038,0 +4039,0 +4040,0 +4041,0 +4042,0 +4043,0 +4044,0 +4045,0 +4046,0 +4047,0 +4048,0 +4049,0 +4050,0 +4051,0 +4052,0 +4053,0 +4054,0 +4055,0 +4056,0 +4057,0 +4058,0 +4059,0 +4060,0 +4061,0 +4062,0 +4063,0 +4064,0 +4065,0 +4066,0 +4067,0 +4068,0 +4069,0 +4070,0 +4071,0 +4072,0 +4073,0 +4074,0 +4075,0 +4076,0 +4077,0 +4078,0 +4079,0 +4080,0 +4081,0 +4082,0 +4083,0 +4084,0 +4085,0 +4086,0 +4087,0 +4088,0 +4089,0 +4090,0 +4091,0 +4092,0 +4093,0 +4094,0 +4095,0 +4096,0 +4097,0 +4098,0 +4099,0 +4100,0 +4101,0 +4102,0 +4103,0 +4104,0 +4105,0 +4106,0 +4107,0 +4108,0 +4109,0 +4110,0 +4111,0 +4112,0 +4113,0 +4114,0 +4115,0 +4116,0 +4117,0 +4118,0 +4119,0 +4120,0 +4121,0 +4122,0 +4123,0 +4124,0 +4125,0 +4126,0 +4127,0 +4128,0 +4129,0 +4130,0 +4131,0 +4132,0 +4133,0 +4134,0 +4135,0 +4136,0 +4137,0 +4138,0 +4139,0 +4140,0 +4141,0 +4142,0 +4143,0 +4144,0 +4145,0 +4146,0 +4147,0 +4148,0 +4149,0 +4150,0 +4151,0 +4152,0 +4153,0 +4154,0 +4155,0 +4156,0 +4157,0 +4158,0 +4159,0 +4160,0 +4161,0 +4162,0 +4163,0 +4164,0 +4165,0 +4166,0 +4167,0 +4168,0 +4169,0 +4170,0 +4171,0 +4172,0 +4173,0 +4174,0 +4175,0 +4176,0 +4177,0 +4178,0 +4179,0 +4180,0 +4181,0 +4182,0 +4183,0 +4184,0 +4185,0 +4186,0 +4187,0 +4188,0 +4189,0 +4190,0 +4191,0 +4192,0 +4193,0 +4194,0 +4195,0 +4196,0 +4197,0 +4198,0 +4199,0 +4200,0 +4201,0 +4202,0 +4203,0 +4204,0 +4205,0 +4206,0 +4207,0 +4208,0 +4209,0 +4210,0 +4211,0 +4212,0 +4213,0 +4214,0 +4215,0 +4216,0 +4217,0 +4218,0 +4219,0 +4220,0 +4221,0 +4222,0 +4223,0 +4224,0 +4225,0 +4226,0 +4227,0 +4228,0 +4229,0 +4230,0 +4231,0 +4232,0 +4233,0 +4234,0 +4235,0 +4236,0 +4237,0 +4238,0 +4239,0 +4240,0 +4241,0 +4242,0 +4243,0 +4244,0 +4245,0 +4246,0 +4247,0 +4248,0 +4249,0 +4250,0 +4251,0 +4252,0 +4253,0 +4254,0 +4255,0 +4256,0 +4257,0 +4258,0 +4259,0 +4260,0 +4261,0 +4262,0 +4263,0 +4264,0 +4265,0 +4266,0 +4267,0 +4268,0 +4269,0 +4270,0 +4271,0 +4272,0 +4273,0 +4274,0 +4275,0 +4276,0 +4277,0 +4278,0 +4279,0 +4280,0 +4281,0 +4282,0 +4283,0 +4284,0 +4285,0 +4286,0 +4287,0 +4288,0 +4289,0 +4290,0 +4291,0 +4292,0 +4293,0 +4294,0 +4295,0 +4296,0 +4297,0 +4298,0 +4299,0 +4300,0 +4301,0 +4302,0 +4303,0 +4304,0 +4305,0 +4306,0 +4307,0 +4308,0 +4309,0 +4310,0 +4311,0 +4312,0 +4313,0 +4314,0 +4315,0 +4316,0 +4317,0 +4318,0 +4319,0 +4320,0 +4321,0 +4322,0 +4323,0 +4324,0 +4325,0 +4326,0 +4327,0 +4328,0 +4329,0 +4330,0 +4331,0 +4332,0 +4333,0 +4334,0 +4335,0 +4336,0 +4337,0 +4338,0 +4339,0 +4340,0 +4341,0 +4342,0 +4343,0 +4344,0 +4345,0 +4346,0 +4347,0 +4348,0 +4349,0 +4350,0 +4351,0 +4352,0 +4353,0 +4354,0 +4355,0 +4356,0 +4357,0 +4358,0 +4359,0 +4360,0 +4361,0 +4362,0 +4363,0 +4364,0 +4365,0 +4366,0 +4367,0 +4368,0 +4369,0 +4370,0 +4371,0 +4372,0 +4373,0 +4374,0 +4375,0 +4376,0 +4377,0 +4378,0 +4379,0 +4380,0 +4381,0 +4382,0 +4383,0 +4384,0 +4385,0 +4386,0 +4387,0 +4388,0 +4389,0 +4390,0 +4391,0 +4392,0 +4393,0 +4394,0 +4395,0 +4396,0 +4397,0 +4398,0 +4399,0 +4400,0 +4401,0 +4402,0 +4403,0 +4404,0 +4405,0 +4406,0 +4407,0 +4408,0 +4409,0 +4410,0 +4411,0 +4412,0 +4413,0 +4414,0 +4415,0 +4416,0 +4417,0 +4418,0 +4419,0 +4420,0 +4421,0 +4422,0 +4423,0 +4424,0 +4425,0 +4426,0 +4427,0 +4428,0 +4429,0 +4430,0 +4431,0 +4432,0 +4433,0 +4434,0 +4435,0 +4436,0 +4437,0 +4438,0 +4439,0 +4440,0 +4441,0 +4442,0 +4443,0 +4444,0 +4445,0 +4446,0 +4447,0 +4448,0 +4449,0 +4450,0 +4451,0 +4452,0 +4453,0 +4454,0 +4455,0 +4456,0 +4457,0 +4458,0 +4459,0 +4460,0 +4461,0 +4462,0 +4463,0 +4464,0 +4465,0 +4466,0 +4467,0 +4468,0 +4469,0 +4470,0 +4471,0 +4472,0 +4473,0 +4474,0 +4475,0 +4476,0 +4477,0 +4478,0 +4479,0 +4480,0 +4481,0 +4482,0 +4483,0 +4484,0 +4485,0 +4486,0 +4487,0 +4488,0 +4489,0 +4490,0 +4491,0 +4492,0 +4493,0 +4494,0 +4495,0 +4496,0 +4497,0 +4498,0 +4499,0 +4500,0 +4501,0 +4502,0 +4503,0 +4504,0 +4505,0 +4506,0 +4507,0 +4508,0 +4509,0 +4510,0 +4511,0 +4512,0 +4513,0 +4514,0 +4515,0 +4516,0 +4517,0 +4518,0 +4519,0 +4520,0 +4521,0 +4522,0 +4523,0 +4524,0 +4525,0 +4526,0 +4527,0 +4528,0 +4529,0 +4530,0 +4531,0 +4532,0 +4533,0 +4534,0 +4535,0 +4536,0 +4537,0 +4538,0 +4539,0 +4540,0 +4541,0 +4542,0 +4543,0 +4544,0 +4545,0 +4546,0 +4547,0 +4548,0 +4549,0 +4550,0 +4551,0 +4552,0 +4553,0 +4554,0 +4555,0 +4556,0 +4557,0 +4558,0 +4559,0 +4560,0 +4561,0 +4562,0 +4563,0 +4564,0 +4565,0 +4566,0 +4567,0 +4568,0 +4569,0 +4570,0 +4571,0 +4572,0 +4573,0 +4574,0 +4575,0 +4576,0 +4577,0 +4578,0 +4579,0 +4580,0 +4581,0 +4582,0 +4583,0 +4584,0 +4585,0 +4586,0 +4587,0 +4588,0 +4589,0 +4590,0 +4591,0 +4592,0 +4593,0 +4594,0 +4595,0 +4596,0 +4597,0 +4598,0 +4599,0 +4600,0 +4601,0 +4602,0 +4603,0 +4604,0 +4605,0 +4606,0 +4607,0 +4608,0 +4609,0 +4610,0 +4611,0 +4612,0 +4613,0 +4614,0 +4615,0 +4616,0 +4617,0 +4618,0 +4619,0 +4620,0 +4621,0 +4622,0 +4623,0 +4624,0 +4625,0 +4626,0 +4627,0 +4628,0 +4629,0 +4630,0 +4631,0 +4632,0 +4633,0 +4634,0 +4635,0 +4636,0 +4637,0 +4638,0 +4639,0 +4640,0 +4641,0 +4642,0 +4643,0 +4644,0 +4645,0 +4646,0 +4647,0 +4648,0 +4649,0 +4650,0 +4651,0 +4652,0 +4653,0 +4654,0 +4655,0 +4656,0 +4657,0 +4658,0 +4659,0 +4660,0 +4661,0 +4662,0 +4663,0 +4664,0 +4665,0 +4666,0 +4667,0 +4668,0 +4669,0 +4670,0 +4671,0 +4672,0 +4673,0 +4674,0 +4675,0 +4676,0 +4677,0 +4678,0 +4679,0 +4680,0 +4681,0 +4682,0 +4683,0 +4684,0 +4685,0 +4686,0 +4687,0 +4688,0 +4689,0 +4690,0 +4691,0 +4692,0 +4693,0 +4694,0 +4695,0 +4696,0 +4697,0 +4698,0 +4699,0 +4700,0 +4701,0 +4702,0 +4703,0 +4704,0 +4705,0 +4706,0 +4707,0 +4708,0 +4709,0 +4710,0 +4711,0 +4712,0 +4713,0 +4714,0 +4715,0 +4716,0 +4717,0 +4718,0 +4719,0 +4720,0 +4721,0 +4722,0 +4723,0 +4724,0 +4725,0 +4726,0 +4727,0 +4728,0 +4729,0 +4730,0 +4731,0 +4732,0 +4733,0 +4734,0 +4735,0 +4736,0 +4737,0 +4738,0 +4739,0 +4740,0 +4741,0 +4742,0 +4743,0 +4744,0 +4745,0 +4746,0 +4747,0 +4748,0 +4749,0 +4750,0 +4751,0 +4752,0 +4753,0 +4754,0 +4755,0 +4756,0 +4757,0 +4758,0 +4759,0 +4760,0 +4761,0 +4762,0 +4763,0 +4764,0 +4765,0 +4766,0 +4767,0 +4768,0 +4769,0 +4770,0 +4771,0 +4772,0 +4773,0 +4774,0 +4775,0 +4776,0 +4777,0 +4778,0 +4779,0 +4780,0 +4781,0 +4782,0 +4783,0 +4784,0 +4785,0 +4786,0 +4787,0 +4788,0 +4789,0 +4790,0 +4791,0 +4792,0 +4793,0 +4794,0 +4795,0 +4796,0 +4797,0 +4798,0 +4799,0 +4800,0 +4801,0 +4802,0 +4803,0 +4804,0 +4805,0 +4806,0 +4807,0 +4808,0 +4809,0 +4810,0 +4811,0 +4812,0 +4813,0 +4814,0 +4815,0 +4816,0 +4817,0 +4818,0 +4819,0 +4820,0 +4821,0 +4822,0 +4823,0 +4824,0 +4825,0 +4826,0 +4827,0 +4828,0 +4829,0 +4830,0 +4831,0 +4832,0 +4833,0 +4834,0 +4835,0 +4836,0 +4837,0 +4838,0 +4839,0 +4840,0 +4841,0 +4842,0 +4843,0 +4844,0 +4845,0 +4846,0 +4847,0 +4848,0 +4849,0 +4850,0 +4851,0 +4852,0 +4853,0 +4854,0 +4855,0 +4856,0 +4857,0 +4858,0 +4859,0 +4860,0 +4861,0 +4862,0 +4863,0 +4864,0 +4865,0 +4866,0 +4867,0 +4868,0 +4869,0 +4870,0 +4871,0 +4872,0 +4873,0 +4874,0 +4875,0 +4876,0 +4877,0 +4878,0 +4879,0 +4880,0 +4881,0 +4882,0 +4883,0 +4884,0 +4885,0 +4886,0 +4887,0 +4888,0 +4889,0 +4890,0 +4891,0 +4892,0 +4893,0 +4894,0 +4895,0 +4896,0 +4897,0 +4898,0 +4899,0 +4900,0 +4901,0 +4902,0 +4903,0 +4904,0 +4905,0 +4906,0 +4907,0 +4908,0 +4909,0 +4910,0 +4911,0 +4912,0 +4913,0 +4914,0 +4915,0 +4916,0 +4917,0 +4918,0 +4919,0 +4920,0 +4921,0 +4922,0 +4923,0 +4924,0 +4925,0 +4926,0 +4927,0 +4928,0 +4929,0 +4930,0 +4931,0 +4932,0 +4933,0 +4934,0 +4935,0 +4936,0 +4937,0 +4938,0 +4939,0 +4940,0 +4941,0 +4942,0 +4943,0 +4944,0 +4945,0 +4946,0 +4947,0 +4948,0 +4949,0 +4950,0 +4951,0 +4952,0 +4953,0 +4954,0 +4955,0 +4956,0 +4957,0 +4958,0 +4959,0 +4960,0 +4961,0 +4962,0 +4963,0 +4964,0 +4965,0 +4966,0 +4967,0 +4968,0 +4969,0 +4970,0 +4971,0 +4972,0 +4973,0 +4974,0 +4975,0 +4976,0 +4977,0 +4978,0 +4979,0 +4980,0 +4981,0 +4982,0 +4983,0 +4984,0 +4985,0 +4986,0 +4987,0 +4988,0 +4989,0 +4990,0 +4991,0 +4992,0 +4993,0 +4994,0 +4995,0 +4996,0 +4997,0 +4998,0 +4999,0 +5000,0 +5001,0 +5002,0 +5003,0 +5004,0 +5005,0 +5006,0 +5007,0 +5008,0 +5009,0 +5010,0 +5011,0 +5012,0 +5013,0 +5014,0 +5015,0 +5016,0 +5017,0 +5018,0 +5019,0 +5020,0 +5021,0 +5022,0 +5023,0 +5024,0 +5025,0 +5026,0 +5027,0 +5028,0 +5029,0 +5030,0 +5031,0 +5032,0 +5033,0 +5034,0 +5035,0 +5036,0 +5037,0 +5038,0 +5039,0 +5040,0 +5041,0 +5042,0 +5043,0 +5044,0 +5045,0 +5046,0 +5047,0 +5048,0 +5049,0 +5050,0 +5051,0 +5052,0 +5053,0 +5054,0 +5055,0 +5056,0 +5057,0 +5058,0 +5059,0 +5060,0 +5061,0 +5062,0 +5063,0 +5064,0 +5065,0 +5066,0 +5067,0 +5068,0 +5069,0 +5070,0 +5071,0 +5072,0 +5073,0 +5074,0 +5075,0 +5076,0 +5077,0 +5078,0 +5079,0 +5080,0 +5081,0 +5082,0 +5083,0 +5084,0 +5085,0 +5086,0 +5087,0 +5088,0 +5089,0 +5090,0 +5091,0 +5092,0 +5093,0 +5094,0 +5095,0 +5096,0 +5097,0 +5098,0 +5099,0 +5100,0 +5101,0 +5102,0 +5103,0 +5104,0 +5105,0 +5106,0 +5107,0 +5108,0 +5109,0 +5110,0 +5111,0 +5112,0 +5113,0 +5114,0 +5115,0 +5116,0 +5117,0 +5118,0 +5119,0 +5120,0 +5121,0 +5122,0 +5123,0 +5124,0 +5125,0 +5126,0 +5127,0 +5128,0 +5129,0 +5130,0 +5131,0 +5132,0 +5133,0 +5134,0 +5135,0 +5136,0 +5137,0 +5138,0 +5139,0 +5140,0 +5141,0 +5142,0 +5143,0 +5144,0 +5145,0 +5146,0 +5147,0 +5148,0 +5149,0 +5150,0 +5151,0 +5152,0 +5153,0 +5154,0 +5155,0 +5156,0 +5157,0 +5158,0 +5159,0 +5160,0 +5161,0 +5162,0 +5163,0 +5164,0 +5165,0 +5166,0 +5167,0 +5168,0 +5169,0 +5170,0 +5171,0 +5172,0 +5173,0 +5174,0 +5175,0 +5176,0 +5177,0 +5178,0 +5179,0 +5180,0 +5181,0 +5182,0 +5183,0 +5184,0 +5185,0 +5186,0 +5187,0 +5188,0 +5189,0 +5190,0 +5191,0 +5192,0 +5193,0 +5194,0 +5195,0 +5196,0 +5197,0 +5198,0 +5199,0 +5200,0 +5201,0 +5202,0 +5203,0 +5204,0 +5205,0 +5206,0 +5207,0 +5208,0 +5209,0 +5210,0 +5211,0 +5212,0 +5213,0 +5214,0 +5215,0 +5216,0 +5217,0 +5218,0 +5219,0 +5220,0 +5221,0 +5222,0 +5223,0 +5224,0 +5225,0 +5226,0 +5227,0 +5228,0 +5229,0 +5230,0 +5231,0 +5232,0 +5233,0 +5234,0 +5235,0 +5236,0 +5237,0 +5238,0 +5239,0 +5240,0 +5241,0 +5242,0 +5243,0 +5244,0 +5245,0 +5246,0 +5247,0 +5248,0 +5249,0 +5250,0 +5251,0 +5252,0 +5253,0 +5254,0 +5255,0 +5256,0 +5257,0 +5258,0 +5259,0 +5260,0 +5261,0 +5262,0 +5263,0 +5264,0 +5265,0 +5266,0 +5267,0 +5268,0 +5269,0 +5270,0 +5271,0 +5272,0 +5273,0 +5274,0 +5275,0 +5276,0 +5277,0 +5278,0 +5279,0 +5280,0 +5281,0 +5282,0 +5283,0 +5284,0 +5285,0 +5286,0 +5287,0 +5288,0 +5289,0 +5290,0 +5291,0 +5292,0 +5293,0 +5294,0 +5295,0 +5296,0 +5297,0 +5298,0 +5299,0 +5300,0 +5301,0 +5302,0 +5303,0 +5304,0 +5305,0 +5306,0 +5307,0 +5308,0 +5309,0 +5310,0 +5311,0 +5312,0 +5313,0 +5314,0 +5315,0 +5316,0 +5317,0 +5318,0 +5319,0 +5320,0 +5321,0 +5322,0 +5323,0 +5324,0 +5325,0 +5326,0 +5327,0 +5328,0 +5329,0 +5330,0 +5331,0 +5332,0 +5333,0 +5334,0 +5335,0 +5336,0 +5337,0 +5338,0 +5339,0 +5340,0 +5341,0 +5342,0 +5343,0 +5344,0 +5345,0 +5346,0 +5347,0 +5348,0 +5349,0 +5350,0 +5351,0 +5352,0 +5353,0 +5354,0 +5355,0 +5356,0 +5357,0 +5358,0 +5359,0 +5360,0 +5361,0 +5362,0 +5363,0 +5364,0 +5365,0 +5366,0 +5367,0 +5368,0 +5369,0 +5370,0 +5371,0 +5372,0 +5373,0 +5374,0 +5375,0 +5376,0 +5377,0 +5378,0 +5379,0 +5380,0 +5381,0 +5382,0 +5383,0 +5384,0 +5385,0 +5386,0 +5387,0 +5388,0 +5389,0 +5390,0 +5391,0 +5392,0 +5393,0 +5394,0 +5395,0 +5396,0 +5397,0 +5398,0 +5399,0 +5400,0 +5401,0 +5402,0 +5403,0 +5404,0 +5405,0 +5406,0 +5407,0 +5408,0 +5409,0 +5410,0 +5411,0 +5412,0 +5413,0 +5414,0 +5415,0 +5416,0 +5417,0 +5418,0 +5419,0 +5420,0 +5421,0 +5422,0 +5423,0 +5424,0 +5425,0 +5426,0 +5427,0 +5428,0 +5429,0 +5430,0 +5431,0 +5432,0 +5433,0 +5434,0 +5435,0 +5436,0 +5437,0 +5438,0 +5439,0 +5440,0 +5441,0 +5442,0 +5443,0 +5444,0 +5445,0 +5446,0 +5447,0 +5448,0 +5449,0 +5450,0 +5451,0 +5452,0 +5453,0 +5454,0 +5455,0 +5456,0 +5457,0 +5458,0 +5459,0 +5460,0 +5461,0 +5462,0 +5463,0 +5464,0 +5465,0 +5466,0 +5467,0 +5468,0 +5469,0 +5470,0 +5471,0 +5472,0 +5473,0 +5474,0 +5475,0 +5476,0 +5477,0 +5478,0 +5479,0 +5480,0 +5481,0 +5482,0 +5483,0 +5484,0 +5485,0 +5486,0 +5487,0 +5488,0 +5489,0 +5490,0 +5491,0 +5492,0 +5493,0 +5494,0 +5495,0 +5496,0 +5497,0 +5498,0 +5499,0 +5500,0 +5501,0 +5502,0 +5503,0 +5504,0 +5505,0 +5506,0 +5507,0 +5508,0 +5509,0 +5510,0 +5511,0 +5512,0 +5513,0 +5514,0 +5515,0 +5516,0 +5517,0 +5518,0 +5519,0 +5520,0 +5521,0 +5522,0 +5523,0 +5524,0 +5525,0 +5526,0 +5527,0 +5528,0 +5529,0 +5530,0 +5531,0 +5532,0 +5533,0 +5534,0 +5535,0 +5536,0 +5537,0 +5538,0 +5539,0 +5540,0 +5541,0 +5542,0 +5543,0 +5544,0 +5545,0 +5546,0 +5547,0 +5548,0 +5549,0 +5550,0 +5551,0 +5552,0 +5553,0 +5554,0 +5555,0 +5556,0 +5557,0 +5558,0 +5559,0 +5560,0 +5561,0 +5562,0 +5563,0 +5564,0 +5565,0 +5566,0 +5567,0 +5568,0 +5569,0 +5570,0 +5571,0 +5572,0 +5573,0 +5574,0 +5575,0 +5576,0 +5577,0 +5578,0 +5579,0 +5580,0 +5581,0 +5582,0 +5583,0 +5584,0 +5585,0 +5586,0 +5587,0 +5588,0 +5589,0 +5590,0 +5591,0 +5592,0 +5593,0 +5594,0 +5595,0 +5596,0 +5597,0 +5598,0 +5599,0 +5600,0 +5601,0 +5602,0 +5603,0 +5604,0 +5605,0 +5606,0 +5607,0 +5608,0 +5609,0 +5610,0 +5611,0 +5612,0 +5613,0 +5614,0 +5615,0 +5616,0 +5617,0 +5618,0 +5619,0 +5620,0 +5621,0 +5622,0 +5623,0 +5624,0 +5625,0 +5626,0 +5627,0 +5628,0 +5629,0 +5630,0 +5631,0 +5632,0 +5633,0 +5634,0 +5635,0 +5636,0 +5637,0 +5638,0 +5639,0 +5640,0 +5641,0 +5642,0 +5643,0 +5644,0 +5645,0 +5646,0 +5647,0 +5648,0 +5649,0 +5650,0 +5651,0 +5652,0 +5653,0 +5654,0 +5655,0 +5656,0 +5657,0 +5658,0 +5659,0 +5660,0 +5661,0 +5662,0 +5663,0 +5664,0 +5665,0 +5666,0 +5667,0 +5668,0 +5669,0 +5670,0 +5671,0 +5672,0 +5673,0 +5674,0 +5675,0 +5676,0 +5677,0 +5678,0 +5679,0 +5680,0 +5681,0 +5682,0 +5683,0 +5684,0 +5685,0 +5686,0 +5687,0 +5688,0 +5689,0 +5690,0 +5691,0 +5692,0 +5693,0 +5694,0 +5695,0 +5696,0 +5697,0 +5698,0 +5699,0 +5700,0 +5701,0 +5702,0 +5703,0 +5704,0 +5705,0 +5706,0 +5707,0 +5708,0 +5709,0 +5710,0 +5711,0 +5712,0 +5713,0 +5714,0 +5715,0 +5716,0 +5717,0 +5718,0 +5719,0 +5720,0 +5721,0 +5722,0 +5723,0 +5724,0 +5725,0 +5726,0 +5727,0 +5728,0 +5729,0 +5730,0 +5731,0 +5732,0 +5733,0 +5734,0 +5735,0 +5736,0 +5737,0 +5738,0 +5739,0 +5740,0 +5741,0 +5742,0 +5743,0 +5744,0 +5745,0 +5746,0 +5747,0 +5748,0 +5749,0 +5750,0 +5751,0 +5752,0 +5753,0 +5754,0 +5755,0 +5756,0 +5757,0 +5758,0 +5759,0 +5760,0 +5761,0 +5762,0 +5763,0 +5764,0 +5765,0 +5766,0 +5767,0 +5768,0 +5769,0 +5770,0 +5771,0 +5772,0 +5773,0 +5774,0 +5775,0 +5776,0 +5777,0 +5778,0 +5779,0 +5780,0 +5781,0 +5782,0 +5783,0 +5784,0 +5785,0 +5786,0 +5787,0 +5788,0 +5789,0 +5790,0 +5791,0 +5792,0 +5793,0 +5794,0 +5795,0 +5796,0 +5797,0 +5798,0 +5799,0 +5800,0 +5801,0 +5802,0 +5803,0 +5804,0 +5805,0 +5806,0 +5807,0 +5808,0 +5809,0 +5810,0 +5811,0 +5812,0 +5813,0 +5814,0 +5815,0 +5816,0 +5817,0 +5818,0 +5819,0 +5820,0 +5821,0 +5822,0 +5823,0 +5824,0 +5825,0 +5826,0 +5827,0 +5828,0 +5829,0 +5830,0 +5831,0 +5832,0 +5833,0 +5834,0 +5835,0 +5836,0 +5837,0 +5838,0 +5839,0 +5840,0 +5841,0 +5842,0 +5843,0 +5844,0 +5845,0 +5846,0 +5847,0 +5848,0 +5849,0 +5850,0 +5851,0 +5852,0 +5853,0 +5854,0 +5855,0 +5856,0 +5857,0 +5858,0 +5859,0 +5860,0 +5861,0 +5862,0 +5863,0 +5864,0 +5865,0 +5866,0 +5867,0 +5868,0 +5869,0 +5870,0 +5871,0 +5872,0 +5873,0 +5874,0 +5875,0 +5876,0 +5877,0 +5878,0 +5879,0 +5880,0 +5881,0 +5882,0 +5883,0 +5884,0 +5885,0 +5886,0 +5887,0 +5888,0 +5889,0 +5890,0 +5891,0 +5892,0 +5893,0 +5894,0 +5895,0 +5896,0 +5897,0 +5898,0 +5899,0 +5900,0 +5901,0 +5902,0 +5903,0 +5904,0 +5905,0 +5906,0 +5907,0 +5908,0 +5909,0 +5910,0 +5911,0 +5912,0 +5913,0 +5914,0 +5915,0 +5916,0 +5917,0 +5918,0 +5919,0 +5920,0 +5921,0 +5922,0 +5923,0 +5924,0 +5925,0 +5926,0 +5927,0 +5928,0 +5929,0 +5930,0 +5931,0 +5932,0 +5933,0 +5934,0 +5935,0 +5936,0 +5937,0 +5938,0 +5939,0 +5940,0 +5941,0 +5942,0 +5943,0 +5944,0 +5945,0 +5946,0 +5947,0 +5948,0 +5949,0 +5950,0 +5951,0 +5952,0 +5953,0 +5954,0 +5955,0 +5956,0 +5957,0 +5958,0 +5959,0 +5960,0 +5961,0 +5962,0 +5963,0 +5964,0 +5965,0 +5966,0 +5967,0 +5968,0 +5969,0 +5970,0 +5971,0 +5972,0 +5973,0 +5974,0 +5975,0 +5976,0 +5977,0 +5978,0 +5979,0 +5980,0 +5981,0 +5982,0 +5983,0 +5984,0 +5985,0 +5986,0 +5987,0 +5988,0 +5989,0 +5990,0 +5991,0 +5992,0 +5993,0 +5994,0 +5995,0 +5996,0 +5997,0 +5998,0 +5999,0 +6000,0 +6001,0 +6002,0 +6003,0 +6004,0 +6005,0 +6006,0 +6007,0 +6008,0 +6009,0 +6010,0 +6011,0 +6012,0 +6013,0 +6014,0 +6015,0 +6016,0 +6017,0 +6018,0 +6019,0 +6020,0 +6021,0 +6022,0 +6023,0 +6024,0 +6025,0 +6026,0 +6027,0 +6028,0 +6029,0 +6030,0 +6031,0 +6032,0 +6033,0 +6034,0 +6035,0 +6036,0 +6037,0 +6038,0 +6039,0 +6040,0 +6041,0 +6042,0 +6043,0 +6044,0 +6045,0 +6046,0 +6047,0 +6048,0 +6049,0 +6050,0 +6051,0 +6052,0 +6053,0 +6054,0 +6055,0 +6056,0 +6057,0 +6058,0 +6059,0 +6060,0 +6061,0 +6062,0 +6063,0 +6064,0 +6065,0 +6066,0 +6067,0 +6068,0 +6069,0 +6070,0 +6071,0 +6072,0 +6073,0 +6074,0 +6075,0 +6076,0 +6077,0 +6078,0 +6079,0 +6080,0 +6081,0 +6082,0 +6083,0 +6084,0 +6085,0 +6086,0 +6087,0 +6088,0 +6089,0 +6090,0 +6091,0 +6092,0 +6093,0 +6094,0 +6095,0 +6096,0 +6097,0 +6098,0 +6099,0 +6100,0 +6101,0 +6102,0 +6103,0 +6104,0 +6105,0 +6106,0 +6107,0 +6108,0 +6109,0 +6110,0 +6111,0 +6112,0 +6113,0 +6114,0 +6115,0 +6116,0 +6117,0 +6118,0 +6119,0 +6120,0 +6121,0 +6122,0 +6123,0 +6124,0 +6125,0 +6126,0 +6127,0 +6128,0 +6129,0 +6130,0 +6131,0 +6132,0 +6133,0 +6134,0 +6135,0 +6136,0 +6137,0 +6138,0 +6139,0 +6140,0 +6141,0 +6142,0 +6143,0 +6144,0 +6145,0 +6146,0 +6147,0 +6148,0 +6149,0 +6150,0 +6151,0 +6152,0 +6153,0 +6154,0 +6155,0 +6156,0 +6157,0 +6158,0 +6159,0 +6160,0 +6161,0 +6162,0 +6163,0 +6164,0 +6165,0 +6166,0 +6167,0 +6168,0 +6169,0 +6170,0 +6171,0 +6172,0 +6173,0 +6174,0 +6175,0 +6176,0 +6177,0 +6178,0 +6179,0 +6180,0 +6181,0 +6182,0 +6183,0 +6184,0 +6185,0 +6186,0 +6187,0 +6188,0 +6189,0 +6190,0 +6191,0 +6192,0 +6193,0 +6194,0 +6195,0 +6196,0 +6197,0 +6198,0 +6199,0 +6200,0 +6201,0 +6202,0 +6203,0 +6204,0 +6205,0 +6206,0 +6207,0 +6208,0 +6209,0 +6210,0 +6211,0 +6212,0 +6213,0 +6214,0 +6215,0 +6216,0 +6217,0 +6218,0 +6219,0 +6220,0 +6221,0 +6222,0 +6223,0 +6224,0 +6225,0 +6226,0 +6227,0 +6228,0 +6229,0 +6230,0 +6231,0 +6232,0 +6233,0 +6234,0 +6235,0 +6236,0 +6237,0 +6238,0 +6239,0 +6240,0 +6241,0 +6242,0 +6243,0 +6244,0 +6245,0 +6246,0 +6247,0 +6248,0 +6249,0 +6250,0 +6251,0 +6252,0 +6253,0 +6254,0 +6255,0 +6256,0 +6257,0 +6258,0 +6259,0 +6260,0 +6261,0 +6262,0 +6263,0 +6264,0 +6265,0 +6266,0 +6267,0 +6268,0 +6269,0 +6270,0 +6271,0 +6272,0 +6273,0 +6274,0 +6275,0 +6276,0 +6277,0 +6278,0 +6279,0 +6280,0 +6281,0 +6282,0 +6283,0 +6284,0 +6285,0 +6286,0 +6287,0 +6288,0 +6289,0 +6290,0 +6291,0 +6292,0 +6293,0 +6294,0 +6295,0 +6296,0 +6297,0 +6298,0 +6299,0 +6300,0 +6301,0 +6302,0 +6303,0 +6304,0 +6305,0 +6306,0 +6307,0 +6308,0 +6309,0 +6310,0 +6311,0 +6312,0 +6313,0 +6314,0 +6315,0 +6316,0 +6317,0 +6318,0 +6319,0 +6320,0 +6321,0 +6322,0 +6323,0 +6324,0 +6325,0 +6326,0 +6327,0 +6328,0 +6329,0 +6330,0 +6331,0 +6332,0 +6333,0 +6334,0 +6335,0 +6336,0 +6337,0 +6338,0 +6339,0 +6340,0 +6341,0 +6342,0 +6343,0 +6344,0 +6345,0 +6346,0 +6347,0 +6348,0 +6349,0 +6350,0 +6351,0 +6352,0 +6353,0 +6354,0 +6355,0 +6356,0 +6357,0 +6358,0 +6359,0 +6360,0 +6361,0 +6362,0 +6363,0 +6364,0 +6365,0 +6366,0 +6367,0 +6368,0 +6369,0 +6370,0 +6371,0 +6372,0 +6373,0 +6374,0 +6375,0 +6376,0 +6377,0 +6378,0 +6379,0 +6380,0 +6381,0 +6382,0 +6383,0 +6384,0 +6385,0 +6386,0 +6387,0 +6388,0 +6389,0 +6390,0 +6391,0 +6392,0 +6393,0 +6394,0 +6395,0 +6396,0 +6397,0 +6398,0 +6399,0 +6400,0 +6401,0 +6402,0 +6403,0 +6404,0 +6405,0 +6406,0 +6407,0 +6408,0 +6409,0 +6410,0 +6411,0 +6412,0 +6413,0 +6414,0 +6415,0 +6416,0 +6417,0 +6418,0 +6419,0 +6420,0 +6421,0 +6422,0 +6423,0 +6424,0 +6425,0 +6426,0 +6427,0 +6428,0 +6429,0 +6430,0 +6431,0 +6432,0 +6433,0 +6434,0 +6435,0 +6436,0 +6437,0 +6438,0 +6439,0 +6440,0 +6441,0 +6442,0 +6443,0 +6444,0 +6445,0 +6446,0 +6447,0 +6448,0 +6449,0 +6450,0 +6451,0 +6452,0 +6453,0 +6454,0 +6455,0 +6456,0 +6457,0 +6458,0 +6459,0 +6460,0 +6461,0 +6462,0 +6463,0 +6464,0 +6465,0 +6466,0 +6467,0 +6468,0 +6469,0 +6470,0 +6471,0 +6472,0 +6473,0 +6474,0 +6475,0 +6476,0 +6477,0 +6478,0 +6479,0 +6480,0 +6481,0 +6482,0 +6483,0 +6484,0 +6485,0 +6486,0 +6487,0 +6488,0 +6489,0 +6490,0 +6491,0 +6492,0 +6493,0 +6494,0 +6495,0 +6496,0 +6497,0 +6498,0 +6499,0 +6500,0 +6501,0 +6502,0 +6503,0 +6504,0 +6505,0 +6506,0 +6507,0 +6508,0 +6509,0 +6510,0 +6511,0 +6512,0 +6513,0 +6514,0 +6515,0 +6516,0 +6517,0 +6518,0 +6519,0 +6520,0 +6521,0 +6522,0 +6523,0 +6524,0 +6525,0 +6526,0 +6527,0 +6528,0 +6529,0 +6530,0 +6531,0 +6532,0 +6533,0 +6534,0 +6535,0 +6536,0 +6537,0 +6538,0 +6539,0 +6540,0 +6541,0 +6542,0 +6543,0 +6544,0 +6545,0 +6546,0 +6547,0 +6548,0 +6549,0 +6550,0 +6551,0 +6552,0 +6553,0 +6554,0 +6555,0 +6556,0 +6557,0 +6558,0 +6559,0 +6560,0 +6561,0 +6562,0 +6563,0 +6564,0 +6565,0 +6566,0 +6567,0 +6568,0 +6569,0 +6570,0 +6571,0 +6572,0 +6573,0 +6574,0 +6575,0 +6576,0 +6577,0 +6578,0 +6579,0 +6580,0 +6581,0 +6582,0 +6583,0 +6584,0 +6585,0 +6586,0 +6587,0 +6588,0 +6589,0 +6590,0 +6591,0 +6592,0 +6593,0 +6594,0 +6595,0 +6596,0 +6597,0 +6598,0 +6599,0 +6600,0 +6601,0 +6602,0 +6603,0 +6604,0 +6605,0 +6606,0 +6607,0 +6608,0 +6609,0 +6610,0 +6611,0 +6612,0 +6613,0 +6614,0 +6615,0 +6616,0 +6617,0 +6618,0 +6619,0 +6620,0 +6621,0 +6622,0 +6623,0 +6624,0 +6625,0 +6626,0 +6627,0 +6628,0 +6629,0 +6630,0 +6631,0 +6632,0 +6633,0 +6634,0 +6635,0 +6636,0 +6637,0 +6638,0 +6639,0 +6640,0 +6641,0 +6642,0 +6643,0 +6644,0 +6645,0 +6646,0 +6647,0 +6648,0 +6649,0 +6650,0 +6651,0 +6652,0 +6653,0 +6654,0 +6655,0 +6656,0 +6657,0 +6658,0 +6659,0 +6660,0 +6661,0 +6662,0 +6663,0 +6664,0 +6665,0 +6666,0 +6667,0 +6668,0 +6669,0 +6670,0 +6671,0 +6672,0 +6673,0 +6674,0 +6675,0 +6676,0 +6677,0 +6678,0 +6679,0 +6680,0 +6681,0 +6682,0 +6683,0 +6684,0 +6685,0 +6686,0 +6687,0 +6688,0 +6689,0 +6690,0 +6691,0 +6692,0 +6693,0 +6694,0 +6695,0 +6696,0 +6697,0 +6698,0 +6699,0 +6700,0 +6701,0 +6702,0 +6703,0 +6704,0 +6705,0 +6706,0 +6707,0 +6708,0 +6709,0 +6710,0 +6711,0 +6712,0 +6713,0 +6714,0 +6715,0 +6716,0 +6717,0 +6718,0 +6719,0 +6720,0 +6721,0 +6722,0 +6723,0 +6724,0 +6725,0 +6726,0 +6727,0 +6728,0 +6729,0 +6730,0 +6731,0 +6732,0 +6733,0 +6734,0 +6735,0 +6736,0 +6737,0 +6738,0 +6739,0 +6740,0 +6741,0 +6742,0 +6743,0 +6744,0 +6745,0 +6746,0 +6747,0 +6748,0 +6749,0 +6750,0 +6751,0 +6752,0 +6753,0 +6754,0 +6755,0 +6756,0 +6757,0 +6758,0 +6759,0 +6760,0 +6761,0 +6762,0 +6763,0 +6764,0 +6765,0 +6766,0 +6767,0 +6768,0 +6769,0 +6770,0 +6771,0 +6772,0 +6773,0 +6774,0 +6775,0 +6776,0 +6777,0 +6778,0 +6779,0 +6780,0 +6781,0 +6782,0 +6783,0 +6784,0 +6785,0 +6786,0 +6787,0 +6788,0 +6789,0 +6790,0 +6791,0 +6792,0 +6793,0 +6794,0 +6795,0 +6796,0 +6797,0 +6798,0 +6799,0 +6800,0 +6801,0 +6802,0 +6803,0 +6804,0 +6805,0 +6806,0 +6807,0 +6808,0 +6809,0 +6810,0 +6811,0 +6812,0 +6813,0 +6814,0 +6815,0 +6816,0 +6817,0 +6818,0 +6819,0 +6820,0 +6821,0 +6822,0 +6823,0 +6824,0 +6825,0 +6826,0 +6827,0 +6828,0 +6829,0 +6830,0 +6831,0 +6832,0 +6833,0 +6834,0 +6835,0 +6836,0 +6837,0 +6838,0 +6839,0 +6840,0 +6841,0 +6842,0 +6843,0 +6844,0 +6845,0 +6846,0 +6847,0 +6848,0 +6849,0 +6850,0 +6851,0 +6852,0 +6853,0 +6854,0 +6855,0 +6856,0 +6857,0 +6858,0 +6859,0 +6860,0 +6861,0 +6862,0 +6863,0 +6864,0 +6865,0 +6866,0 +6867,0 +6868,0 +6869,0 +6870,0 +6871,0 +6872,0 +6873,0 +6874,0 +6875,0 +6876,0 +6877,0 +6878,0 +6879,0 +6880,0 +6881,0 +6882,0 +6883,0 +6884,0 +6885,0 +6886,0 +6887,0 +6888,0 +6889,0 +6890,0 +6891,0 +6892,0 +6893,0 +6894,0 +6895,0 +6896,0 +6897,0 +6898,0 +6899,0 +6900,0 +6901,0 +6902,0 +6903,0 +6904,0 +6905,0 +6906,0 +6907,0 +6908,0 +6909,0 +6910,0 +6911,0 +6912,0 +6913,0 +6914,0 +6915,0 +6916,0 +6917,0 +6918,0 +6919,0 +6920,0 +6921,0 +6922,0 +6923,0 +6924,0 +6925,0 +6926,0 +6927,0 +6928,0 +6929,0 +6930,0 +6931,0 +6932,0 +6933,0 +6934,0 +6935,0 +6936,0 +6937,0 +6938,0 +6939,0 +6940,0 +6941,0 +6942,0 +6943,0 +6944,0 +6945,0 +6946,0 +6947,0 +6948,0 +6949,0 +6950,0 +6951,0 +6952,0 +6953,0 +6954,0 +6955,0 +6956,0 +6957,0 +6958,0 +6959,0 +6960,0 +6961,0 +6962,0 +6963,0 +6964,0 +6965,0 +6966,0 +6967,0 +6968,0 +6969,0 +6970,0 +6971,0 +6972,0 +6973,0 +6974,0 +6975,0 +6976,0 +6977,0 +6978,0 +6979,0 +6980,0 +6981,0 +6982,0 +6983,0 +6984,0 +6985,0 +6986,0 +6987,0 +6988,0 +6989,0 +6990,0 +6991,0 +6992,0 +6993,0 +6994,0 +6995,0 +6996,0 +6997,0 +6998,0 +6999,0 +7000,0 +7001,0 +7002,0 +7003,0 +7004,0 +7005,0 +7006,0 +7007,0 +7008,0 +7009,0 +7010,0 +7011,0 +7012,0 +7013,0 +7014,0 +7015,0 +7016,0 +7017,0 +7018,0 +7019,0 +7020,0 +7021,0 +7022,0 +7023,0 +7024,0 +7025,0 +7026,0 +7027,0 +7028,0 +7029,0 +7030,0 +7031,0 +7032,0 +7033,0 +7034,0 +7035,0 +7036,0 +7037,0 +7038,0 +7039,0 +7040,0 +7041,0 +7042,0 +7043,0 +7044,0 +7045,0 +7046,0 +7047,0 +7048,0 +7049,0 +7050,0 +7051,0 +7052,0 +7053,0 +7054,0 +7055,0 +7056,0 +7057,0 +7058,0 +7059,0 +7060,0 +7061,0 +7062,0 +7063,0 +7064,0 +7065,0 +7066,0 +7067,0 +7068,0 +7069,0 +7070,0 +7071,0 +7072,0 +7073,0 +7074,0 +7075,0 +7076,0 +7077,0 +7078,0 +7079,0 +7080,0 +7081,0 +7082,0 +7083,0 +7084,0 +7085,0 +7086,0 +7087,0 +7088,0 +7089,0 +7090,0 +7091,0 +7092,0 +7093,0 +7094,0 +7095,0 +7096,0 +7097,0 +7098,0 +7099,0 +7100,0 +7101,0 +7102,0 +7103,0 +7104,0 +7105,0 +7106,0 +7107,0 +7108,0 +7109,0 +7110,0 +7111,0 +7112,0 +7113,0 +7114,0 +7115,0 +7116,0 +7117,0 +7118,0 +7119,0 +7120,0 +7121,0 +7122,0 +7123,0 +7124,0 +7125,0 +7126,0 +7127,0 +7128,0 +7129,0 +7130,0 +7131,0 +7132,0 +7133,0 +7134,0 +7135,0 +7136,0 +7137,0 +7138,0 +7139,0 +7140,0 +7141,0 +7142,0 +7143,0 +7144,0 +7145,0 +7146,0 +7147,0 +7148,0 +7149,0 +7150,0 +7151,0 +7152,0 +7153,0 +7154,0 +7155,0 +7156,0 +7157,0 +7158,0 +7159,0 +7160,0 +7161,0 +7162,0 +7163,0 +7164,0 +7165,0 +7166,0 +7167,0 +7168,0 +7169,0 +7170,0 +7171,0 +7172,0 +7173,0 +7174,0 +7175,0 +7176,0 +7177,0 +7178,0 +7179,0 +7180,0 +7181,0 +7182,0 +7183,0 +7184,0 +7185,0 +7186,0 +7187,0 +7188,0 +7189,0 +7190,0 +7191,0 +7192,0 +7193,0 +7194,0 +7195,0 +7196,0 +7197,0 +7198,0 +7199,0 +7200,0 +7201,0 +7202,0 +7203,0 +7204,0 +7205,0 +7206,0 +7207,0 +7208,0 +7209,0 +7210,0 +7211,0 +7212,0 +7213,0 +7214,0 +7215,0 +7216,0 +7217,0 +7218,0 +7219,0 +7220,0 +7221,0 +7222,0 +7223,0 +7224,0 +7225,0 +7226,0 +7227,0 +7228,0 +7229,0 +7230,0 +7231,0 +7232,0 +7233,0 +7234,0 +7235,0 +7236,0 +7237,0 +7238,0 +7239,0 +7240,0 +7241,0 +7242,0 +7243,0 +7244,0 +7245,0 +7246,0 +7247,0 +7248,0 +7249,0 +7250,0 +7251,0 +7252,0 +7253,0 +7254,0 +7255,0 +7256,0 +7257,0 +7258,0 +7259,0 +7260,0 +7261,0 +7262,0 +7263,0 +7264,0 +7265,0 +7266,0 +7267,0 +7268,0 +7269,0 +7270,0 +7271,0 +7272,0 +7273,0 +7274,0 +7275,0 +7276,0 +7277,0 +7278,0 +7279,0 +7280,0 +7281,0 +7282,0 +7283,0 +7284,0 +7285,0 +7286,0 +7287,0 +7288,0 +7289,0 +7290,0 +7291,0 +7292,0 +7293,0 +7294,0 +7295,0 +7296,0 +7297,0 +7298,0 +7299,0 +7300,0 +7301,0 +7302,0 +7303,0 +7304,0 +7305,0 +7306,0 +7307,0 +7308,0 +7309,0 +7310,0 +7311,0 +7312,0 +7313,0 +7314,0 +7315,0 +7316,0 +7317,0 +7318,0 +7319,0 +7320,0 +7321,0 +7322,0 +7323,0 +7324,0 +7325,0 +7326,0 +7327,0 +7328,0 +7329,0 +7330,0 +7331,0 +7332,0 +7333,0 +7334,0 +7335,0 +7336,0 +7337,0 +7338,0 +7339,0 +7340,0 +7341,0 +7342,0 +7343,0 +7344,0 +7345,0 +7346,0 +7347,0 +7348,0 +7349,0 +7350,0 +7351,0 +7352,0 +7353,0 +7354,0 +7355,0 +7356,0 +7357,0 +7358,0 +7359,0 +7360,0 +7361,0 +7362,0 +7363,0 +7364,0 +7365,0 +7366,0 +7367,0 +7368,0 +7369,0 +7370,0 +7371,0 +7372,0 +7373,0 +7374,0 +7375,0 +7376,0 +7377,0 +7378,0 +7379,0 +7380,0 +7381,0 +7382,0 +7383,0 +7384,0 +7385,0 +7386,0 +7387,0 +7388,0 +7389,0 +7390,0 +7391,0 +7392,0 +7393,0 +7394,0 +7395,0 +7396,0 +7397,0 +7398,0 +7399,0 +7400,0 +7401,0 +7402,0 +7403,0 +7404,0 +7405,0 +7406,0 +7407,0 +7408,0 +7409,0 +7410,0 +7411,0 +7412,0 +7413,0 +7414,0 +7415,0 +7416,0 +7417,0 +7418,0 +7419,0 +7420,0 +7421,0 +7422,0 +7423,0 +7424,0 +7425,0 +7426,0 +7427,0 +7428,0 +7429,0 +7430,0 +7431,0 +7432,0 +7433,0 +7434,0 +7435,0 +7436,0 +7437,0 +7438,0 +7439,0 +7440,0 +7441,0 +7442,0 +7443,0 +7444,0 +7445,0 +7446,0 +7447,0 +7448,0 +7449,0 +7450,0 +7451,0 +7452,0 +7453,0 +7454,0 +7455,0 +7456,0 +7457,0 +7458,0 +7459,0 +7460,0 +7461,0 +7462,0 +7463,0 +7464,0 +7465,0 +7466,0 +7467,0 +7468,0 +7469,0 +7470,0 +7471,0 +7472,0 +7473,0 +7474,0 +7475,0 +7476,0 +7477,0 +7478,0 +7479,0 +7480,0 +7481,0 +7482,0 +7483,0 +7484,0 +7485,0 +7486,0 +7487,0 +7488,0 +7489,0 +7490,0 +7491,0 +7492,0 +7493,0 +7494,0 +7495,0 +7496,0 +7497,0 +7498,0 +7499,0 +7500,0 +7501,0 +7502,0 +7503,0 +7504,0 +7505,0 +7506,0 +7507,0 +7508,0 +7509,0 +7510,0 +7511,0 +7512,0 +7513,0 +7514,0 +7515,0 +7516,0 +7517,0 +7518,0 +7519,0 +7520,0 +7521,0 +7522,0 +7523,0 +7524,0 +7525,0 +7526,0 +7527,0 +7528,0 +7529,0 +7530,0 +7531,0 +7532,0 +7533,0 +7534,0 +7535,0 +7536,0 +7537,0 +7538,0 +7539,0 +7540,0 +7541,0 +7542,0 +7543,0 +7544,0 +7545,0 +7546,0 +7547,0 +7548,0 +7549,0 +7550,0 +7551,0 +7552,0 +7553,0 +7554,0 +7555,0 +7556,0 +7557,0 +7558,0 +7559,0 +7560,0 +7561,0 +7562,0 +7563,0 +7564,0 +7565,0 +7566,0 +7567,0 +7568,0 +7569,0 +7570,0 +7571,0 +7572,0 +7573,0 +7574,0 +7575,0 +7576,0 +7577,0 +7578,0 +7579,0 +7580,0 +7581,0 +7582,0 +7583,0 +7584,0 +7585,0 +7586,0 +7587,0 +7588,0 +7589,0 +7590,0 +7591,0 +7592,0 +7593,0 +7594,0 +7595,0 +7596,0 +7597,0 +7598,0 +7599,0 +7600,0 +7601,0 +7602,0 +7603,0 +7604,0 +7605,0 +7606,0 +7607,0 +7608,0 +7609,0 +7610,0 +7611,0 +7612,0 +7613,0 +7614,0 +7615,0 +7616,0 +7617,0 +7618,0 +7619,0 +7620,0 +7621,0 +7622,0 +7623,0 +7624,0 +7625,0 +7626,0 +7627,0 +7628,0 +7629,0 +7630,0 +7631,0 +7632,0 +7633,0 +7634,0 +7635,0 +7636,0 +7637,0 +7638,0 +7639,0 +7640,0 +7641,0 +7642,0 +7643,0 +7644,0 +7645,0 +7646,0 +7647,0 +7648,0 +7649,0 +7650,0 +7651,0 +7652,0 +7653,0 +7654,0 +7655,0 +7656,0 +7657,0 +7658,0 +7659,0 +7660,0 +7661,0 +7662,0 +7663,0 +7664,0 +7665,0 +7666,0 +7667,0 +7668,0 +7669,0 +7670,0 +7671,0 +7672,0 +7673,0 +7674,0 +7675,0 +7676,0 +7677,0 +7678,0 +7679,0 +7680,0 +7681,0 +7682,0 +7683,0 +7684,0 +7685,0 +7686,0 +7687,0 +7688,0 +7689,0 +7690,0 +7691,0 +7692,0 +7693,0 +7694,0 +7695,0 +7696,0 +7697,0 +7698,0 +7699,0 +7700,0 +7701,0 +7702,0 +7703,0 +7704,0 +7705,0 +7706,0 +7707,0 +7708,0 +7709,0 +7710,0 +7711,0 +7712,0 +7713,0 +7714,0 +7715,0 +7716,0 +7717,0 +7718,0 +7719,0 +7720,0 +7721,0 +7722,0 +7723,0 +7724,0 +7725,0 +7726,0 +7727,0 +7728,0 +7729,0 +7730,0 +7731,0 +7732,0 +7733,0 +7734,0 +7735,0 +7736,0 +7737,0 +7738,0 +7739,0 +7740,0 +7741,0 +7742,0 +7743,0 +7744,0 +7745,0 +7746,0 +7747,0 +7748,0 +7749,0 +7750,0 +7751,0 +7752,0 +7753,0 +7754,0 +7755,0 +7756,0 +7757,0 +7758,0 +7759,0 +7760,0 +7761,0 +7762,0 +7763,0 +7764,0 +7765,0 +7766,0 +7767,0 +7768,0 +7769,0 +7770,0 +7771,0 +7772,0 +7773,0 +7774,0 +7775,0 +7776,0 +7777,0 +7778,0 +7779,0 +7780,0 +7781,0 +7782,0 +7783,0 +7784,0 +7785,0 +7786,0 +7787,0 +7788,0 +7789,0 +7790,0 +7791,0 +7792,0 +7793,0 +7794,0 +7795,0 +7796,0 +7797,0 +7798,0 +7799,0 +7800,0 +7801,0 +7802,0 +7803,0 +7804,0 +7805,0 +7806,0 +7807,0 +7808,0 +7809,0 +7810,0 +7811,0 +7812,0 +7813,0 +7814,0 +7815,0 +7816,0 +7817,0 +7818,0 +7819,0 +7820,0 +7821,0 +7822,0 +7823,0 +7824,0 +7825,0 +7826,0 +7827,0 +7828,0 +7829,0 +7830,0 +7831,0 +7832,0 +7833,0 +7834,0 +7835,0 +7836,0 +7837,0 +7838,0 +7839,0 +7840,0 +7841,0 +7842,0 +7843,0 +7844,0 +7845,0 +7846,0 +7847,0 +7848,0 +7849,0 +7850,0 +7851,0 +7852,0 +7853,0 +7854,0 +7855,0 +7856,0 +7857,0 +7858,0 +7859,0 +7860,0 +7861,0 +7862,0 +7863,0 +7864,0 +7865,0 +7866,0 +7867,0 +7868,0 +7869,0 +7870,0 +7871,0 +7872,0 +7873,0 +7874,0 +7875,0 +7876,0 +7877,0 +7878,0 +7879,0 +7880,0 +7881,0 +7882,0 +7883,0 +7884,0 +7885,0 +7886,0 +7887,0 +7888,0 +7889,0 +7890,0 +7891,0 +7892,0 +7893,0 +7894,0 +7895,0 +7896,0 +7897,0 +7898,0 +7899,0 +7900,0 +7901,0 +7902,0 +7903,0 +7904,0 +7905,0 +7906,0 +7907,0 +7908,0 +7909,0 +7910,0 +7911,0 +7912,0 +7913,0 +7914,0 +7915,0 +7916,0 +7917,0 +7918,0 +7919,0 +7920,0 +7921,0 +7922,0 +7923,0 +7924,0 +7925,0 +7926,0 +7927,0 +7928,0 +7929,0 +7930,0 +7931,0 +7932,0 +7933,0 +7934,0 +7935,0 +7936,0 +7937,0 +7938,0 +7939,0 +7940,0 +7941,0 +7942,0 +7943,0 +7944,0 +7945,0 +7946,0 +7947,0 +7948,0 +7949,0 +7950,0 +7951,0 +7952,0 +7953,0 +7954,0 +7955,0 +7956,0 +7957,0 +7958,0 +7959,0 +7960,0 +7961,0 +7962,0 +7963,0 +7964,0 +7965,0 +7966,0 +7967,0 +7968,0 +7969,0 +7970,0 +7971,0 +7972,0 +7973,0 +7974,0 +7975,0 +7976,0 +7977,0 +7978,0 +7979,0 +7980,0 +7981,0 +7982,0 +7983,0 +7984,0 +7985,0 +7986,0 +7987,0 +7988,0 +7989,0 +7990,0 +7991,0 +7992,0 +7993,0 +7994,0 +7995,0 +7996,0 +7997,0 +7998,0 +7999,0 +8000,0 +8001,0 +8002,0 +8003,0 +8004,0 +8005,0 +8006,0 +8007,0 +8008,0 +8009,0 +8010,0 +8011,0 +8012,0 +8013,0 +8014,0 +8015,0 +8016,0 +8017,0 +8018,0 +8019,0 +8020,0 +8021,0 +8022,0 +8023,0 +8024,0 +8025,0 +8026,0 +8027,0 +8028,0 +8029,0 +8030,0 +8031,0 +8032,0 +8033,0 +8034,0 +8035,0 +8036,0 +8037,0 +8038,0 +8039,0 +8040,0 +8041,0 +8042,0 +8043,0 +8044,0 +8045,0 +8046,0 +8047,0 +8048,0 +8049,0 +8050,0 +8051,0 +8052,0 +8053,0 +8054,0 +8055,0 +8056,0 +8057,0 +8058,0 +8059,0 +8060,0 +8061,0 +8062,0 +8063,0 +8064,0 +8065,0 +8066,0 +8067,0 +8068,0 +8069,0 +8070,0 +8071,0 +8072,0 +8073,0 +8074,0 +8075,0 +8076,0 +8077,0 +8078,0 +8079,0 +8080,0 +8081,0 +8082,0 +8083,0 +8084,0 +8085,0 +8086,0 +8087,0 +8088,0 +8089,0 +8090,0 +8091,0 +8092,0 +8093,0 +8094,0 +8095,0 +8096,0 +8097,0 +8098,0 +8099,0 +8100,0 +8101,0 +8102,0 +8103,0 +8104,0 +8105,0 +8106,0 +8107,0 +8108,0 +8109,0 +8110,0 +8111,0 +8112,0 +8113,0 +8114,0 +8115,0 +8116,0 +8117,0 +8118,0 +8119,0 +8120,0 +8121,0 +8122,0 +8123,0 +8124,0 +8125,0 +8126,0 +8127,0 +8128,0 +8129,0 +8130,0 +8131,0 +8132,0 +8133,0 +8134,0 +8135,0 +8136,0 +8137,0 +8138,0 +8139,0 +8140,0 +8141,0 +8142,0 +8143,0 +8144,0 +8145,0 +8146,0 +8147,0 +8148,0 +8149,0 +8150,0 +8151,0 +8152,0 +8153,0 +8154,0 +8155,0 +8156,0 +8157,0 +8158,0 +8159,0 +8160,0 +8161,0 +8162,0 +8163,0 +8164,0 +8165,0 +8166,0 +8167,0 +8168,0 +8169,0 +8170,0 +8171,0 +8172,0 +8173,0 +8174,0 +8175,0 +8176,0 +8177,0 +8178,0 +8179,0 +8180,0 +8181,0 +8182,0 +8183,0 +8184,0 +8185,0 +8186,0 +8187,0 +8188,0 +8189,0 +8190,0 +8191,0 +8192,0 +8193,0 +8194,0 +8195,0 +8196,0 +8197,0 +8198,0 +8199,0 +8200,0 +8201,0 +8202,0 +8203,0 +8204,0 +8205,0 +8206,0 +8207,0 +8208,0 +8209,0 +8210,0 +8211,0 +8212,0 +8213,0 +8214,0 +8215,0 +8216,0 +8217,0 +8218,0 +8219,0 +8220,0 +8221,0 +8222,0 +8223,0 +8224,0 +8225,0 +8226,0 +8227,0 +8228,0 +8229,0 +8230,0 +8231,0 +8232,0 +8233,0 +8234,0 +8235,0 +8236,0 +8237,0 +8238,0 +8239,0 +8240,0 +8241,0 +8242,0 +8243,0 +8244,0 +8245,0 +8246,0 +8247,0 +8248,0 +8249,0 +8250,0 +8251,0 +8252,0 +8253,0 +8254,0 +8255,0 +8256,0 +8257,0 +8258,0 +8259,0 +8260,0 +8261,0 +8262,0 +8263,0 +8264,0 +8265,0 +8266,0 +8267,0 +8268,0 +8269,0 +8270,0 +8271,0 +8272,0 +8273,0 +8274,0 +8275,0 +8276,0 +8277,0 +8278,0 +8279,0 +8280,0 +8281,0 +8282,0 +8283,0 +8284,0 +8285,0 +8286,0 +8287,0 +8288,0 +8289,0 +8290,0 +8291,0 +8292,0 +8293,0 +8294,0 +8295,0 +8296,0 +8297,0 +8298,0 +8299,0 +8300,0 +8301,0 +8302,0 +8303,0 +8304,0 +8305,0 +8306,0 +8307,0 +8308,0 +8309,0 +8310,0 +8311,0 +8312,0 +8313,0 +8314,0 +8315,0 +8316,0 +8317,0 +8318,0 +8319,0 +8320,0 +8321,0 +8322,0 +8323,0 +8324,0 +8325,0 +8326,0 +8327,0 +8328,0 +8329,0 +8330,0 +8331,0 +8332,0 +8333,0 +8334,0 +8335,0 +8336,0 +8337,0 +8338,0 +8339,0 +8340,0 +8341,0 +8342,0 +8343,0 +8344,0 +8345,0 +8346,0 +8347,0 +8348,0 +8349,0 +8350,0 +8351,0 +8352,0 +8353,0 +8354,0 +8355,0 +8356,0 +8357,0 +8358,0 +8359,0 +8360,0 +8361,0 +8362,0 +8363,0 +8364,0 +8365,0 +8366,0 +8367,0 +8368,0 +8369,0 +8370,0 +8371,0 +8372,0 +8373,0 +8374,0 +8375,0 +8376,0 +8377,0 +8378,0 +8379,0 +8380,0 +8381,0 +8382,0 +8383,0 +8384,0 +8385,0 +8386,0 +8387,0 +8388,0 +8389,0 +8390,0 +8391,0 +8392,0 +8393,0 +8394,0 +8395,0 +8396,0 +8397,0 +8398,0 +8399,0 +8400,0 +8401,0 +8402,0 +8403,0 +8404,0 +8405,0 +8406,0 +8407,0 +8408,0 +8409,0 +8410,0 +8411,0 +8412,0 +8413,0 +8414,0 +8415,0 +8416,0 +8417,0 +8418,0 +8419,0 +8420,0 +8421,0 +8422,0 +8423,0 +8424,0 +8425,0 +8426,0 +8427,0 +8428,0 +8429,0 +8430,0 +8431,0 +8432,0 +8433,0 +8434,0 +8435,0 +8436,0 +8437,0 +8438,0 +8439,0 +8440,0 +8441,0 +8442,0 +8443,0 +8444,0 +8445,0 +8446,0 +8447,0 +8448,0 +8449,0 +8450,0 +8451,0 +8452,0 +8453,0 +8454,0 +8455,0 +8456,0 +8457,0 +8458,0 +8459,0 +8460,0 +8461,0 +8462,0 +8463,0 +8464,0 +8465,0 +8466,0 +8467,0 +8468,0 +8469,0 +8470,0 +8471,0 +8472,0 +8473,0 +8474,0 +8475,0 +8476,0 +8477,0 +8478,0 +8479,0 +8480,0 +8481,0 +8482,0 +8483,0 +8484,0 +8485,0 +8486,0 +8487,0 +8488,0 +8489,0 +8490,0 +8491,0 +8492,0 +8493,0 +8494,0 +8495,0 +8496,0 +8497,0 +8498,0 +8499,0 +8500,0 +8501,0 +8502,0 +8503,0 +8504,0 +8505,0 +8506,0 +8507,0 +8508,0 +8509,0 +8510,0 +8511,0 +8512,0 +8513,0 +8514,0 +8515,0 +8516,0 +8517,0 +8518,0 +8519,0 +8520,0 +8521,0 +8522,0 +8523,0 +8524,0 +8525,0 +8526,0 +8527,0 +8528,0 +8529,0 +8530,0 +8531,0 +8532,0 +8533,0 +8534,0 +8535,0 +8536,0 +8537,0 +8538,0 +8539,0 +8540,0 +8541,0 +8542,0 +8543,0 +8544,0 +8545,0 +8546,0 +8547,0 +8548,0 +8549,0 +8550,0 +8551,0 +8552,0 +8553,0 +8554,0 +8555,0 +8556,0 +8557,0 +8558,0 +8559,0 +8560,0 +8561,0 +8562,0 +8563,0 +8564,0 +8565,0 +8566,0 +8567,0 +8568,0 +8569,0 +8570,0 +8571,0 +8572,0 +8573,0 +8574,0 +8575,0 +8576,0 +8577,0 +8578,0 +8579,0 +8580,0 +8581,0 +8582,0 +8583,0 +8584,0 +8585,0 +8586,0 +8587,0 +8588,0 +8589,0 +8590,0 +8591,0 +8592,0 +8593,0 +8594,0 +8595,0 +8596,0 +8597,0 +8598,0 +8599,0 +8600,0 +8601,0 +8602,0 +8603,0 +8604,0 +8605,0 +8606,0 +8607,0 +8608,0 +8609,0 +8610,0 +8611,0 +8612,0 +8613,0 +8614,0 +8615,0 +8616,0 +8617,0 +8618,0 +8619,0 +8620,0 +8621,0 +8622,0 +8623,0 +8624,0 +8625,0 +8626,0 +8627,0 +8628,0 +8629,0 +8630,0 +8631,0 +8632,0 +8633,0 +8634,0 +8635,0 +8636,0 +8637,0 +8638,0 +8639,0 +8640,0 +8641,0 +8642,0 +8643,0 +8644,0 +8645,0 +8646,0 +8647,0 +8648,0 +8649,0 +8650,0 +8651,0 +8652,0 +8653,0 +8654,0 +8655,0 +8656,0 +8657,0 +8658,0 +8659,0 +8660,0 +8661,0 +8662,0 +8663,0 +8664,0 +8665,0 +8666,0 +8667,0 +8668,0 +8669,0 +8670,0 +8671,0 +8672,0 +8673,0 +8674,0 +8675,0 +8676,0 +8677,0 +8678,0 +8679,0 +8680,0 +8681,0 +8682,0 +8683,0 +8684,0 +8685,0 +8686,0 +8687,0 +8688,0 +8689,0 +8690,0 +8691,0 +8692,0 +8693,0 +8694,0 +8695,0 +8696,0 +8697,0 +8698,0 +8699,0 +8700,0 +8701,0 +8702,0 +8703,0 +8704,0 +8705,0 +8706,0 +8707,0 +8708,0 +8709,0 +8710,0 +8711,0 +8712,0 +8713,0 +8714,0 +8715,0 +8716,0 +8717,0 +8718,0 +8719,0 +8720,0 +8721,0 +8722,0 +8723,0 +8724,0 +8725,0 +8726,0 +8727,0 +8728,0 +8729,0 +8730,0 +8731,0 +8732,0 +8733,0 +8734,0 +8735,0 +8736,0 +8737,0 +8738,0 +8739,0 +8740,0 +8741,0 +8742,0 +8743,0 +8744,0 +8745,0 +8746,0 +8747,0 +8748,0 +8749,0 +8750,0 +8751,0 +8752,0 +8753,0 +8754,0 +8755,0 +8756,0 +8757,0 +8758,0 +8759,0 +8760,0 +8761,0 +8762,0 +8763,0 +8764,0 +8765,0 +8766,0 +8767,0 +8768,0 +8769,0 +8770,0 +8771,0 +8772,0 +8773,0 +8774,0 +8775,0 +8776,0 +8777,0 +8778,0 +8779,0 +8780,0 +8781,0 +8782,0 +8783,0 +8784,0 +8785,0 +8786,0 +8787,0 +8788,0 +8789,0 +8790,0 +8791,0 +8792,0 +8793,0 +8794,0 +8795,0 +8796,0 +8797,0 +8798,0 +8799,0 +8800,0 +8801,0 +8802,0 +8803,0 +8804,0 +8805,0 +8806,0 +8807,0 +8808,0 +8809,0 +8810,0 +8811,0 +8812,0 +8813,0 +8814,0 +8815,0 +8816,0 +8817,0 +8818,0 +8819,0 +8820,0 +8821,0 +8822,0 +8823,0 +8824,0 +8825,0 +8826,0 +8827,0 +8828,0 +8829,0 +8830,0 +8831,0 +8832,0 +8833,0 +8834,0 +8835,0 +8836,0 +8837,0 +8838,0 +8839,0 +8840,0 +8841,0 +8842,0 +8843,0 +8844,0 +8845,0 +8846,0 +8847,0 +8848,0 +8849,0 +8850,0 +8851,0 +8852,0 +8853,0 +8854,0 +8855,0 +8856,0 +8857,0 +8858,0 +8859,0 +8860,0 +8861,0 +8862,0 +8863,0 +8864,0 +8865,0 +8866,0 +8867,0 +8868,0 +8869,0 +8870,0 +8871,0 +8872,0 +8873,0 +8874,0 +8875,0 +8876,0 +8877,0 +8878,0 +8879,0 +8880,0 +8881,0 +8882,0 +8883,0 +8884,0 +8885,0 +8886,0 +8887,0 +8888,0 +8889,0 +8890,0 +8891,0 +8892,0 +8893,0 +8894,0 +8895,0 +8896,0 +8897,0 +8898,0 +8899,0 +8900,0 +8901,0 +8902,0 +8903,0 +8904,0 +8905,0 +8906,0 +8907,0 +8908,0 +8909,0 +8910,0 +8911,0 +8912,0 +8913,0 +8914,0 +8915,0 +8916,0 +8917,0 +8918,0 +8919,0 +8920,0 +8921,0 +8922,0 +8923,0 +8924,0 +8925,0 +8926,0 +8927,0 +8928,0 +8929,0 +8930,0 +8931,0 +8932,0 +8933,0 +8934,0 +8935,0 +8936,0 +8937,0 +8938,0 +8939,0 +8940,0 +8941,0 +8942,0 +8943,0 +8944,0 +8945,0 +8946,0 +8947,0 +8948,0 +8949,0 +8950,0 +8951,0 +8952,0 +8953,0 +8954,0 +8955,0 +8956,0 +8957,0 +8958,0 +8959,0 +8960,0 +8961,0 +8962,0 +8963,0 +8964,0 +8965,0 +8966,0 +8967,0 +8968,0 +8969,0 +8970,0 +8971,0 +8972,0 +8973,0 +8974,0 +8975,0 +8976,0 +8977,0 +8978,0 +8979,0 +8980,0 +8981,0 +8982,0 +8983,0 +8984,0 +8985,0 +8986,0 +8987,0 +8988,0 +8989,0 +8990,0 +8991,0 +8992,0 +8993,0 +8994,0 +8995,0 +8996,0 +8997,0 +8998,0 +8999,0 +9000,0 +9001,0 +9002,0 +9003,0 +9004,0 +9005,0 +9006,0 +9007,0 +9008,0 +9009,0 +9010,0 +9011,0 +9012,0 +9013,0 +9014,0 +9015,0 +9016,0 +9017,0 +9018,0 +9019,0 +9020,0 +9021,0 +9022,0 +9023,0 +9024,0 +9025,0 +9026,0 +9027,0 +9028,0 +9029,0 +9030,0 +9031,0 +9032,0 +9033,0 +9034,0 +9035,0 +9036,0 +9037,0 +9038,0 +9039,0 +9040,0 +9041,0 +9042,0 +9043,0 +9044,0 +9045,0 +9046,0 +9047,0 +9048,0 +9049,0 +9050,0 +9051,0 +9052,0 +9053,0 +9054,0 +9055,0 +9056,0 +9057,0 +9058,0 +9059,0 +9060,0 +9061,0 +9062,0 +9063,0 +9064,0 +9065,0 +9066,0 +9067,0 +9068,0 +9069,0 +9070,0 +9071,0 +9072,0 +9073,0 +9074,0 +9075,0 +9076,0 +9077,0 +9078,0 +9079,0 +9080,0 +9081,0 +9082,0 +9083,0 +9084,0 +9085,0 +9086,0 +9087,0 +9088,0 +9089,0 +9090,0 +9091,0 +9092,0 +9093,0 +9094,0 +9095,0 +9096,0 +9097,0 +9098,0 +9099,0 +9100,0 +9101,0 +9102,0 +9103,0 +9104,0 +9105,0 +9106,0 +9107,0 +9108,0 +9109,0 +9110,0 +9111,0 +9112,0 +9113,0 +9114,0 +9115,0 +9116,0 +9117,0 +9118,0 +9119,0 +9120,0 +9121,0 +9122,0 +9123,0 +9124,0 +9125,0 +9126,0 +9127,0 +9128,0 +9129,0 +9130,0 +9131,0 +9132,0 +9133,0 +9134,0 +9135,0 +9136,0 +9137,0 +9138,0 +9139,0 +9140,0 +9141,0 +9142,0 +9143,0 +9144,0 +9145,0 +9146,0 +9147,0 +9148,0 +9149,0 +9150,0 +9151,0 +9152,0 +9153,0 +9154,0 +9155,0 +9156,0 +9157,0 +9158,0 +9159,0 +9160,0 +9161,0 +9162,0 +9163,0 +9164,0 +9165,0 +9166,0 +9167,0 +9168,0 +9169,0 +9170,0 +9171,0 +9172,0 +9173,0 +9174,0 +9175,0 +9176,0 +9177,0 +9178,0 +9179,0 +9180,0 +9181,0 +9182,0 +9183,0 +9184,0 +9185,0 +9186,0 +9187,0 +9188,0 +9189,0 +9190,0 +9191,0 +9192,0 +9193,0 +9194,0 +9195,0 +9196,0 +9197,0 +9198,0 +9199,0 +9200,0 +9201,0 +9202,0 +9203,0 +9204,0 +9205,0 +9206,0 +9207,0 +9208,0 +9209,0 +9210,0 +9211,0 +9212,0 +9213,0 +9214,0 +9215,0 +9216,0 +9217,0 +9218,0 +9219,0 +9220,0 +9221,0 +9222,0 +9223,0 +9224,0 +9225,0 +9226,0 +9227,0 +9228,0 +9229,0 +9230,0 +9231,0 +9232,0 +9233,0 +9234,0 +9235,0 +9236,0 +9237,0 +9238,0 +9239,0 +9240,0 +9241,0 +9242,0 +9243,0 +9244,0 +9245,0 +9246,0 +9247,0 +9248,0 +9249,0 +9250,0 +9251,0 +9252,0 +9253,0 +9254,0 +9255,0 +9256,0 +9257,0 +9258,0 +9259,0 +9260,0 +9261,0 +9262,0 +9263,0 +9264,0 +9265,0 +9266,0 +9267,0 +9268,0 +9269,0 +9270,0 +9271,0 +9272,0 +9273,0 +9274,0 +9275,0 +9276,0 +9277,0 +9278,0 +9279,0 +9280,0 +9281,0 +9282,0 +9283,0 +9284,0 +9285,0 +9286,0 +9287,0 +9288,0 +9289,0 +9290,0 +9291,0 +9292,0 +9293,0 +9294,0 +9295,0 +9296,0 +9297,0 +9298,0 +9299,0 +9300,0 +9301,0 +9302,0 +9303,0 +9304,0 +9305,0 +9306,0 +9307,0 +9308,0 +9309,0 +9310,0 +9311,0 +9312,0 +9313,0 +9314,0 +9315,0 +9316,0 +9317,0 +9318,0 +9319,0 +9320,0 +9321,0 +9322,0 +9323,0 +9324,0 +9325,0 +9326,0 +9327,0 +9328,0 +9329,0 +9330,0 +9331,0 +9332,0 +9333,0 +9334,0 +9335,0 +9336,0 +9337,0 +9338,0 +9339,0 +9340,0 +9341,0 +9342,0 +9343,0 +9344,0 +9345,0 +9346,0 +9347,0 +9348,0 +9349,0 +9350,0 +9351,0 +9352,0 +9353,0 +9354,0 +9355,0 +9356,0 +9357,0 +9358,0 +9359,0 +9360,0 +9361,0 +9362,0 +9363,0 +9364,0 +9365,0 +9366,0 +9367,0 +9368,0 +9369,0 +9370,0 +9371,0 +9372,0 +9373,0 +9374,0 +9375,0 +9376,0 +9377,0 +9378,0 +9379,0 +9380,0 +9381,0 +9382,0 +9383,0 +9384,0 +9385,0 +9386,0 +9387,0 +9388,0 +9389,0 +9390,0 +9391,0 +9392,0 +9393,0 +9394,0 +9395,0 +9396,0 +9397,0 +9398,0 +9399,0 +9400,0 +9401,0 +9402,0 +9403,0 +9404,0 +9405,0 +9406,0 +9407,0 +9408,0 +9409,0 +9410,0 +9411,0 +9412,0 +9413,0 +9414,0 +9415,0 +9416,0 +9417,0 +9418,0 +9419,0 +9420,0 +9421,0 +9422,0 +9423,0 +9424,0 +9425,0 +9426,0 +9427,0 +9428,0 +9429,0 +9430,0 +9431,0 +9432,0 +9433,0 +9434,0 +9435,0 +9436,0 +9437,0 +9438,0 +9439,0 +9440,0 +9441,0 +9442,0 +9443,0 +9444,0 +9445,0 +9446,0 +9447,0 +9448,0 +9449,0 +9450,0 +9451,0 +9452,0 +9453,0 +9454,0 +9455,0 +9456,0 +9457,0 +9458,0 +9459,0 +9460,0 +9461,0 +9462,0 +9463,0 +9464,0 +9465,0 +9466,0 +9467,0 +9468,0 +9469,0 +9470,0 +9471,0 +9472,0 +9473,0 +9474,0 +9475,0 +9476,0 +9477,0 +9478,0 +9479,0 +9480,0 +9481,0 +9482,0 +9483,0 +9484,0 +9485,0 +9486,0 +9487,0 +9488,0 +9489,0 +9490,0 +9491,0 +9492,0 +9493,0 +9494,0 +9495,0 +9496,0 +9497,0 +9498,0 +9499,0 +9500,0 +9501,0 +9502,0 +9503,0 +9504,0 +9505,0 +9506,0 +9507,0 +9508,0 +9509,0 +9510,0 +9511,0 +9512,0 +9513,0 +9514,0 +9515,0 +9516,0 +9517,0 +9518,0 +9519,0 +9520,0 +9521,0 +9522,0 +9523,0 +9524,0 +9525,0 +9526,0 +9527,0 +9528,0 +9529,0 +9530,0 +9531,0 +9532,0 +9533,0 +9534,0 +9535,0 +9536,0 +9537,0 +9538,0 +9539,0 +9540,0 +9541,0 +9542,0 +9543,0 +9544,0 +9545,0 +9546,0 +9547,0 +9548,0 +9549,0 +9550,0 +9551,0 +9552,0 +9553,0 +9554,0 +9555,0 +9556,0 +9557,0 +9558,0 +9559,0 +9560,0 +9561,0 +9562,0 +9563,0 +9564,0 +9565,0 +9566,0 +9567,0 +9568,0 +9569,0 +9570,0 +9571,0 +9572,0 +9573,0 +9574,0 +9575,0 +9576,0 +9577,0 +9578,0 +9579,0 +9580,0 +9581,0 +9582,0 +9583,0 +9584,0 +9585,0 +9586,0 +9587,0 +9588,0 +9589,0 +9590,0 +9591,0 +9592,0 +9593,0 +9594,0 +9595,0 +9596,0 +9597,0 +9598,0 +9599,0 +9600,0 +9601,0 +9602,0 +9603,0 +9604,0 +9605,0 +9606,0 +9607,0 +9608,0 +9609,0 +9610,0 +9611,0 +9612,0 +9613,0 +9614,0 +9615,0 +9616,0 +9617,0 +9618,0 +9619,0 +9620,0 +9621,0 +9622,0 +9623,0 +9624,0 +9625,0 +9626,0 +9627,0 +9628,0 +9629,0 +9630,0 +9631,0 +9632,0 +9633,0 +9634,0 +9635,0 +9636,0 +9637,0 +9638,0 +9639,0 +9640,0 +9641,0 +9642,0 +9643,0 +9644,0 +9645,0 +9646,0 +9647,0 +9648,0 +9649,0 +9650,0 +9651,0 +9652,0 +9653,0 +9654,0 +9655,0 +9656,0 +9657,0 +9658,0 +9659,0 +9660,0 +9661,0 +9662,0 +9663,0 +9664,0 +9665,0 +9666,0 +9667,0 +9668,0 +9669,0 +9670,0 +9671,0 +9672,0 +9673,0 +9674,0 +9675,0 +9676,0 +9677,0 +9678,0 +9679,0 +9680,0 +9681,0 +9682,0 +9683,0 +9684,0 +9685,0 +9686,0 +9687,0 +9688,0 +9689,0 +9690,0 +9691,0 +9692,0 +9693,0 +9694,0 +9695,0 +9696,0 +9697,0 +9698,0 +9699,0 +9700,0 +9701,0 +9702,0 +9703,0 +9704,0 +9705,0 +9706,0 +9707,0 +9708,0 +9709,0 +9710,0 +9711,0 +9712,0 +9713,0 +9714,0 +9715,0 +9716,0 +9717,0 +9718,0 +9719,0 +9720,0 +9721,0 +9722,0 +9723,0 +9724,0 +9725,0 +9726,0 +9727,0 +9728,0 +9729,0 +9730,0 +9731,0 +9732,0 +9733,0 +9734,0 +9735,0 +9736,0 +9737,0 +9738,0 +9739,0 +9740,0 +9741,0 +9742,0 +9743,0 +9744,0 +9745,0 +9746,0 +9747,0 +9748,0 +9749,0 +9750,0 +9751,0 +9752,0 +9753,0 +9754,0 +9755,0 +9756,0 +9757,0 +9758,0 +9759,0 +9760,0 +9761,0 +9762,0 +9763,0 +9764,0 +9765,0 +9766,0 +9767,0 +9768,0 +9769,0 +9770,0 +9771,0 +9772,0 +9773,0 +9774,0 +9775,0 +9776,0 +9777,0 +9778,0 +9779,0 +9780,0 +9781,0 +9782,0 +9783,0 +9784,0 +9785,0 +9786,0 +9787,0 +9788,0 +9789,0 +9790,0 +9791,0 +9792,0 +9793,0 +9794,0 +9795,0 +9796,0 +9797,0 +9798,0 +9799,0 +9800,0 +9801,0 +9802,0 +9803,0 +9804,0 +9805,0 +9806,0 +9807,0 +9808,0 +9809,0 +9810,0 +9811,0 +9812,0 +9813,0 +9814,0 +9815,0 +9816,0 +9817,0 +9818,0 +9819,0 +9820,0 +9821,0 +9822,0 +9823,0 +9824,0 +9825,0 +9826,0 +9827,0 +9828,0 +9829,0 +9830,0 +9831,0 +9832,0 +9833,0 +9834,0 +9835,0 +9836,0 +9837,0 +9838,0 +9839,0 +9840,0 +9841,0 +9842,0 +9843,0 +9844,0 +9845,0 +9846,0 +9847,0 +9848,0 +9849,0 +9850,0 +9851,0 +9852,0 +9853,0 +9854,0 +9855,0 +9856,0 +9857,0 +9858,0 +9859,0 +9860,0 +9861,0 +9862,0 +9863,0 +9864,0 +9865,0 +9866,0 +9867,0 +9868,0 +9869,0 +9870,0 +9871,0 +9872,0 +9873,0 +9874,0 +9875,0 +9876,0 +9877,0 +9878,0 +9879,0 +9880,0 +9881,0 +9882,0 +9883,0 +9884,0 +9885,0 +9886,0 +9887,0 +9888,0 +9889,0 +9890,0 +9891,0 +9892,0 +9893,0 +9894,0 +9895,0 +9896,0 +9897,0 +9898,0 +9899,0 +9900,0 +9901,0 +9902,0 +9903,0 +9904,0 +9905,0 +9906,0 +9907,0 +9908,0 +9909,0 +9910,0 +9911,0 +9912,0 +9913,0 +9914,0 +9915,0 +9916,0 +9917,0 +9918,0 +9919,0 +9920,0 +9921,0 +9922,0 +9923,0 +9924,0 +9925,0 +9926,0 +9927,0 +9928,0 +9929,0 +9930,0 +9931,0 +9932,0 +9933,0 +9934,0 +9935,0 +9936,0 +9937,0 +9938,0 +9939,0 +9940,0 +9941,0 +9942,0 +9943,0 +9944,0 +9945,0 +9946,0 +9947,0 +9948,0 +9949,0 +9950,0 +9951,0 +9952,0 +9953,0 +9954,0 +9955,0 +9956,0 +9957,0 +9958,0 +9959,0 +9960,0 +9961,0 +9962,0 +9963,0 +9964,0 +9965,0 +9966,0 +9967,0 +9968,0 +9969,0 +9970,0 +9971,0 +9972,0 +9973,0 +9974,0 +9975,0 +9976,0 +9977,0 +9978,0 +9979,0 +9980,0 +9981,0 +9982,0 +9983,0 +9984,0 +9985,0 +9986,0 +9987,0 +9988,0 +9989,0 +9990,0 +9991,0 +9992,0 +9993,0 +9994,0 +9995,0 +9996,0 +9997,0 +9998,0 +9999,0 +10000,0 +10001,0 +10002,0 +10003,0 +10004,0 +10005,0 +10006,0 +10007,0 +10008,0 +10009,0 +10010,0 +10011,0 +10012,0 +10013,0 +10014,0 +10015,0 +10016,0 +10017,0 +10018,0 +10019,0 +10020,0 +10021,0 +10022,0 +10023,0 +10024,0 +10025,0 +10026,0 +10027,0 +10028,0 +10029,0 +10030,0 +10031,0 +10032,0 +10033,0 +10034,0 +10035,0 +10036,0 +10037,0 +10038,0 +10039,0 +10040,0 +10041,0 +10042,0 +10043,0 +10044,0 +10045,0 +10046,0 +10047,0 +10048,0 +10049,0 +10050,0 +10051,0 +10052,0 +10053,0 +10054,0 +10055,0 +10056,0 +10057,0 +10058,0 +10059,0 +10060,0 +10061,0 +10062,0 +10063,0 +10064,0 +10065,0 +10066,0 +10067,0 +10068,0 +10069,0 +10070,0 +10071,0 +10072,0 +10073,0 +10074,0 +10075,0 +10076,0 +10077,0 +10078,0 +10079,0 +10080,0 +10081,0 +10082,0 +10083,0 +10084,0 +10085,0 +10086,0 +10087,0 +10088,0 +10089,0 +10090,0 +10091,0 +10092,0 +10093,0 +10094,0 +10095,0 +10096,0 +10097,0 +10098,0 +10099,0 +10100,0 +10101,0 +10102,0 +10103,0 +10104,0 +10105,0 +10106,0 +10107,0 +10108,0 +10109,0 +10110,0 +10111,0 +10112,0 +10113,0 +10114,0 +10115,0 +10116,0 +10117,0 +10118,0 +10119,0 +10120,0 +10121,0 +10122,0 +10123,0 +10124,0 +10125,0 +10126,0 +10127,0 +10128,0 +10129,0 +10130,0 +10131,0 +10132,0 +10133,0 +10134,0 +10135,0 +10136,0 +10137,0 +10138,0 +10139,0 +10140,0 +10141,0 +10142,0 +10143,0 +10144,0 +10145,0 +10146,0 +10147,0 +10148,0 +10149,0 +10150,0 +10151,0 +10152,0 +10153,0 +10154,0 +10155,0 +10156,0 +10157,0 +10158,0 +10159,0 +10160,0 +10161,0 +10162,0 +10163,0 +10164,0 +10165,0 +10166,0 +10167,0 +10168,0 +10169,0 +10170,0 +10171,0 +10172,0 +10173,0 +10174,0 +10175,0 +10176,0 +10177,0 +10178,0 +10179,0 +10180,0 +10181,0 +10182,0 +10183,0 +10184,0 +10185,0 +10186,0 +10187,0 +10188,0 +10189,0 +10190,0 +10191,0 +10192,0 +10193,0 +10194,0 +10195,0 +10196,0 +10197,0 +10198,0 +10199,0 +10200,0 +10201,0 +10202,0 +10203,0 +10204,0 +10205,0 +10206,0 +10207,0 +10208,0 +10209,0 +10210,0 +10211,0 +10212,0 +10213,0 +10214,0 +10215,0 +10216,0 +10217,0 +10218,0 +10219,0 +10220,0 +10221,0 +10222,0 +10223,0 +10224,0 +10225,0 +10226,0 +10227,0 +10228,0 +10229,0 +10230,0 +10231,0 +10232,0 +10233,0 +10234,0 +10235,0 +10236,0 +10237,0 +10238,0 +10239,0 +10240,0 +10241,0 +10242,0 +10243,0 +10244,0 +10245,0 +10246,0 +10247,0 +10248,0 +10249,0 +10250,0 +10251,0 +10252,0 +10253,0 +10254,0 +10255,0 +10256,0 +10257,0 +10258,0 +10259,0 +10260,0 +10261,0 +10262,0 +10263,0 +10264,0 +10265,0 +10266,0 +10267,0 +10268,0 +10269,0 +10270,0 +10271,0 +10272,0 +10273,0 +10274,0 +10275,0 +10276,0 +10277,0 +10278,0 +10279,0 +10280,0 +10281,0 +10282,0 +10283,0 +10284,0 +10285,0 +10286,0 +10287,0 +10288,0 +10289,0 +10290,0 +10291,0 +10292,0 +10293,0 +10294,0 +10295,0 +10296,0 +10297,0 +10298,0 +10299,0 +10300,0 +10301,0 +10302,0 +10303,0 +10304,0 +10305,0 +10306,0 +10307,0 +10308,0 +10309,0 +10310,0 +10311,0 +10312,0 +10313,0 +10314,0 +10315,0 +10316,0 +10317,0 +10318,0 +10319,0 +10320,0 +10321,0 +10322,0 +10323,0 +10324,0 +10325,0 +10326,0 +10327,0 +10328,0 +10329,0 +10330,0 +10331,0 +10332,0 +10333,0 +10334,0 +10335,0 +10336,0 +10337,0 +10338,0 +10339,0 +10340,0 +10341,0 +10342,0 +10343,0 +10344,0 +10345,0 +10346,0 +10347,0 +10348,0 +10349,0 +10350,0 +10351,0 +10352,0 +10353,0 +10354,0 +10355,0 +10356,0 +10357,0 +10358,0 +10359,0 +10360,0 +10361,0 +10362,0 +10363,0 +10364,0 +10365,0 +10366,0 +10367,0 +10368,0 +10369,0 +10370,0 +10371,0 +10372,0 +10373,0 +10374,0 +10375,0 +10376,0 +10377,0 +10378,0 +10379,0 +10380,0 +10381,0 +10382,0 +10383,0 +10384,0 +10385,0 +10386,0 +10387,0 +10388,0 +10389,0 +10390,0 +10391,0 +10392,0 +10393,0 +10394,0 +10395,0 +10396,0 +10397,0 +10398,0 +10399,0 +10400,0 +10401,0 +10402,0 +10403,0 +10404,0 +10405,0 +10406,0 +10407,0 +10408,0 +10409,0 +10410,0 +10411,0 +10412,0 +10413,0 +10414,0 +10415,0 +10416,0 +10417,0 +10418,0 +10419,0 +10420,0 +10421,0 +10422,0 +10423,0 +10424,0 +10425,0 +10426,0 +10427,0 +10428,0 +10429,0 +10430,0 +10431,0 +10432,0 +10433,0 +10434,0 +10435,0 +10436,0 +10437,0 +10438,0 +10439,0 +10440,0 +10441,0 +10442,0 +10443,0 +10444,0 +10445,0 +10446,0 +10447,0 +10448,0 +10449,0 +10450,0 +10451,0 +10452,0 +10453,0 +10454,0 +10455,0 +10456,0 +10457,0 +10458,0 +10459,0 +10460,0 +10461,0 +10462,0 +10463,0 +10464,0 +10465,0 +10466,0 +10467,0 +10468,0 +10469,0 +10470,0 +10471,0 +10472,0 +10473,0 +10474,0 +10475,0 +10476,0 +10477,0 +10478,0 +10479,0 +10480,0 +10481,0 +10482,0 +10483,0 +10484,0 +10485,0 +10486,0 +10487,0 +10488,0 +10489,0 +10490,0 +10491,0 +10492,0 +10493,0 +10494,0 +10495,0 +10496,0 +10497,0 +10498,0 +10499,0 +10500,0 +10501,0 +10502,0 +10503,0 +10504,0 +10505,0 +10506,0 +10507,0 +10508,0 +10509,0 +10510,0 +10511,0 +10512,0 +10513,0 +10514,0 +10515,0 +10516,0 +10517,0 +10518,0 +10519,0 +10520,0 +10521,0 +10522,0 +10523,0 +10524,0 +10525,0 +10526,0 +10527,0 +10528,0 +10529,0 +10530,0 +10531,0 +10532,0 +10533,0 +10534,0 +10535,0 +10536,0 +10537,0 +10538,0 +10539,0 +10540,0 +10541,0 +10542,0 +10543,0 +10544,0 +10545,0 +10546,0 +10547,0 +10548,0 +10549,0 +10550,0 +10551,0 +10552,0 +10553,0 +10554,0 +10555,0 +10556,0 +10557,0 +10558,0 +10559,0 +10560,0 +10561,0 +10562,0 +10563,0 +10564,0 +10565,0 +10566,0 +10567,0 +10568,0 +10569,0 +10570,0 +10571,0 +10572,0 +10573,0 +10574,0 +10575,0 +10576,0 +10577,0 +10578,0 +10579,0 +10580,0 +10581,0 +10582,0 +10583,0 +10584,0 +10585,0 +10586,0 +10587,0 +10588,0 +10589,0 +10590,0 +10591,0 +10592,0 +10593,0 +10594,0 +10595,0 +10596,0 +10597,0 +10598,0 +10599,0 +10600,0 +10601,0 +10602,0 +10603,0 +10604,0 +10605,0 +10606,0 +10607,0 +10608,0 +10609,0 +10610,0 +10611,0 +10612,0 +10613,0 +10614,0 +10615,0 +10616,0 +10617,0 +10618,0 +10619,0 +10620,0 +10621,0 +10622,0 +10623,0 +10624,0 +10625,0 +10626,0 +10627,0 +10628,0 +10629,0 +10630,0 +10631,0 +10632,0 +10633,0 +10634,0 +10635,0 +10636,0 +10637,0 +10638,0 +10639,0 +10640,0 +10641,0 +10642,0 +10643,0 +10644,0 +10645,0 +10646,0 +10647,0 +10648,0 +10649,0 +10650,0 +10651,0 +10652,0 +10653,0 +10654,0 +10655,0 +10656,0 +10657,0 +10658,0 +10659,0 +10660,0 +10661,0 +10662,0 +10663,0 +10664,0 +10665,0 +10666,0 +10667,0 +10668,0 +10669,0 +10670,0 +10671,0 +10672,0 +10673,0 +10674,0 +10675,0 +10676,0 +10677,0 +10678,0 +10679,0 +10680,0 +10681,0 +10682,0 +10683,0 +10684,0 +10685,0 +10686,0 +10687,0 +10688,0 +10689,0 +10690,0 +10691,0 +10692,0 +10693,0 +10694,0 +10695,0 +10696,0 +10697,0 +10698,0 +10699,0 +10700,0 +10701,0 +10702,0 +10703,0 +10704,0 +10705,0 +10706,0 +10707,0 +10708,0 +10709,0 +10710,0 +10711,0 +10712,0 +10713,0 +10714,0 +10715,0 +10716,0 +10717,0 +10718,0 +10719,0 +10720,0 +10721,0 +10722,0 +10723,0 +10724,0 +10725,0 +10726,0 +10727,0 +10728,0 +10729,0 +10730,0 +10731,0 +10732,0 +10733,0 +10734,0 +10735,0 +10736,0 +10737,0 +10738,0 +10739,0 +10740,0 +10741,0 +10742,0 +10743,0 +10744,0 +10745,0 +10746,0 +10747,0 +10748,0 +10749,0 +10750,0 +10751,0 +10752,0 +10753,0 +10754,0 +10755,0 +10756,0 +10757,0 +10758,0 +10759,0 +10760,0 +10761,0 +10762,0 +10763,0 +10764,0 +10765,0 +10766,0 +10767,0 +10768,0 +10769,0 +10770,0 +10771,0 +10772,0 +10773,0 +10774,0 +10775,0 +10776,0 +10777,0 +10778,0 +10779,0 +10780,0 +10781,0 +10782,0 +10783,0 +10784,0 +10785,0 +10786,0 +10787,0 +10788,0 +10789,0 +10790,0 +10791,0 +10792,0 +10793,0 +10794,0 +10795,0 +10796,0 +10797,0 +10798,0 +10799,0 +10800,0 +10801,0 +10802,0 +10803,0 +10804,0 +10805,0 +10806,0 +10807,0 +10808,0 +10809,0 +10810,0 +10811,0 +10812,0 +10813,0 +10814,0 +10815,0 +10816,0 +10817,0 +10818,0 +10819,0 +10820,0 +10821,0 +10822,0 +10823,0 +10824,0 +10825,0 +10826,0 +10827,0 +10828,0 +10829,0 +10830,0 +10831,0 +10832,0 +10833,0 +10834,0 +10835,0 +10836,0 +10837,0 +10838,0 +10839,0 +10840,0 +10841,0 +10842,0 +10843,0 +10844,0 +10845,0 +10846,0 +10847,0 +10848,0 +10849,0 +10850,0 +10851,0 +10852,0 +10853,0 +10854,0 +10855,0 +10856,0 +10857,0 +10858,0 +10859,0 +10860,0 +10861,0 +10862,0 +10863,0 +10864,0 +10865,0 +10866,0 +10867,0 +10868,0 +10869,0 +10870,0 +10871,0 +10872,0 +10873,0 +10874,0 +10875,0 +10876,0 +10877,0 +10878,0 +10879,0 +10880,0 +10881,0 +10882,0 +10883,0 +10884,0 +10885,0 +10886,0 +10887,0 +10888,0 +10889,0 +10890,0 +10891,0 +10892,0 +10893,0 +10894,0 +10895,0 +10896,0 +10897,0 +10898,0 +10899,0 +10900,0 +10901,0 +10902,0 +10903,0 +10904,0 +10905,0 +10906,0 +10907,0 +10908,0 +10909,0 +10910,0 +10911,0 +10912,0 +10913,0 +10914,0 +10915,0 +10916,0 +10917,0 +10918,0 +10919,0 +10920,0 +10921,0 +10922,0 +10923,0 +10924,0 +10925,0 +10926,0 +10927,0 +10928,0 +10929,0 +10930,0 +10931,0 +10932,0 +10933,0 +10934,0 +10935,0 +10936,0 +10937,0 +10938,0 +10939,0 +10940,0 +10941,0 +10942,0 +10943,0 +10944,0 +10945,0 +10946,0 +10947,0 +10948,0 +10949,0 +10950,0 +10951,0 +10952,0 +10953,0 +10954,0 +10955,0 +10956,0 +10957,0 +10958,0 +10959,0 +10960,0 +10961,0 +10962,0 +10963,0 +10964,0 +10965,0 +10966,0 +10967,0 +10968,0 +10969,0 +10970,0 +10971,0 +10972,0 +10973,0 +10974,0 +10975,0 +10976,0 +10977,0 +10978,0 +10979,0 +10980,0 +10981,0 +10982,0 +10983,0 +10984,0 +10985,0 +10986,0 +10987,0 +10988,0 +10989,0 +10990,0 +10991,0 +10992,0 +10993,0 +10994,0 +10995,0 +10996,0 +10997,0 +10998,0 +10999,0 +11000,0 +11001,0 +11002,0 +11003,0 +11004,0 +11005,0 +11006,0 +11007,0 +11008,0 +11009,0 +11010,0 +11011,0 +11012,0 +11013,0 +11014,0 +11015,0 +11016,0 +11017,0 +11018,0 +11019,0 +11020,0 +11021,0 +11022,0 +11023,0 +11024,0 +11025,0 +11026,0 +11027,0 +11028,0 +11029,0 +11030,0 +11031,0 +11032,0 +11033,0 +11034,0 +11035,0 +11036,0 +11037,0 +11038,0 +11039,0 +11040,0 +11041,0 +11042,0 +11043,0 +11044,0 +11045,0 +11046,0 +11047,0 +11048,0 +11049,0 +11050,0 +11051,0 +11052,0 +11053,0 +11054,0 +11055,0 +11056,0 +11057,0 +11058,0 +11059,0 +11060,0 +11061,0 +11062,0 +11063,0 +11064,0 +11065,0 +11066,0 +11067,0 +11068,0 +11069,0 +11070,0 +11071,0 +11072,0 +11073,0 +11074,0 +11075,0 +11076,0 +11077,0 +11078,0 +11079,0 +11080,0 +11081,0 +11082,0 +11083,0 +11084,0 +11085,0 +11086,0 +11087,0 +11088,0 +11089,0 +11090,0 +11091,0 +11092,0 +11093,0 +11094,0 +11095,0 +11096,0 +11097,0 +11098,0 +11099,0 +11100,0 +11101,0 +11102,0 +11103,0 +11104,0 +11105,0 +11106,0 +11107,0 +11108,0 +11109,0 +11110,0 +11111,0 +11112,0 +11113,0 +11114,0 +11115,0 +11116,0 +11117,0 +11118,0 +11119,0 +11120,0 +11121,0 +11122,0 +11123,0 +11124,0 +11125,0 +11126,0 +11127,0 +11128,0 +11129,0 +11130,0 +11131,0 +11132,0 +11133,0 +11134,0 +11135,0 +11136,0 +11137,0 +11138,0 +11139,0 +11140,0 +11141,0 +11142,0 +11143,0 +11144,0 +11145,0 +11146,0 +11147,0 +11148,0 +11149,0 +11150,0 +11151,0 +11152,0 +11153,0 +11154,0 +11155,0 +11156,0 +11157,0 +11158,0 +11159,0 +11160,0 +11161,0 +11162,0 +11163,0 +11164,0 +11165,0 +11166,0 +11167,0 +11168,0 +11169,0 +11170,0 +11171,0 +11172,0 +11173,0 +11174,0 +11175,0 +11176,0 +11177,0 +11178,0 +11179,0 +11180,0 +11181,0 +11182,0 +11183,0 +11184,0 +11185,0 +11186,0 +11187,0 +11188,0 +11189,0 +11190,0 +11191,0 +11192,0 +11193,0 +11194,0 +11195,0 +11196,0 +11197,0 +11198,0 +11199,0 +11200,0 +11201,0 +11202,0 +11203,0 +11204,0 +11205,0 +11206,0 +11207,0 +11208,0 +11209,0 +11210,0 +11211,0 +11212,0 +11213,0 +11214,0 +11215,0 +11216,0 +11217,0 +11218,0 +11219,0 +11220,0 +11221,0 +11222,0 +11223,0 +11224,0 +11225,0 +11226,0 +11227,0 +11228,0 +11229,0 +11230,0 +11231,0 +11232,0 +11233,0 +11234,0 +11235,0 +11236,0 +11237,0 +11238,0 +11239,0 +11240,0 +11241,0 +11242,0 +11243,0 +11244,0 +11245,0 +11246,0 +11247,0 +11248,0 +11249,0 +11250,0 +11251,0 +11252,0 +11253,0 +11254,0 +11255,0 +11256,0 +11257,0 +11258,0 +11259,0 +11260,0 +11261,0 +11262,0 +11263,0 +11264,0 +11265,0 +11266,0 +11267,0 +11268,0 +11269,0 +11270,0 +11271,0 +11272,0 +11273,0 +11274,0 +11275,0 +11276,0 +11277,0 +11278,0 +11279,0 +11280,0 +11281,0 +11282,0 +11283,0 +11284,0 +11285,0 +11286,0 +11287,0 +11288,0 +11289,0 +11290,0 +11291,0 +11292,0 +11293,0 +11294,0 +11295,0 +11296,0 +11297,0 +11298,0 +11299,0 +11300,0 +11301,0 +11302,0 +11303,0 +11304,0 +11305,0 +11306,0 +11307,0 +11308,0 +11309,0 +11310,0 +11311,0 +11312,0 +11313,0 +11314,0 +11315,0 +11316,0 +11317,0 +11318,0 +11319,0 +11320,0 +11321,0 +11322,0 +11323,0 +11324,0 +11325,0 +11326,0 +11327,0 +11328,0 +11329,0 +11330,0 +11331,0 +11332,0 +11333,0 +11334,0 +11335,0 +11336,0 +11337,0 +11338,0 +11339,0 +11340,0 +11341,0 +11342,0 +11343,0 +11344,0 +11345,0 +11346,0 +11347,0 +11348,0 +11349,0 +11350,0 +11351,0 +11352,0 +11353,0 +11354,0 +11355,0 +11356,0 +11357,0 +11358,0 +11359,0 +11360,0 +11361,0 +11362,0 +11363,0 +11364,0 +11365,0 +11366,0 +11367,0 +11368,0 +11369,0 +11370,0 +11371,0 +11372,0 +11373,0 +11374,0 +11375,0 +11376,0 +11377,0 +11378,0 +11379,0 +11380,0 +11381,0 +11382,0 +11383,0 +11384,0 +11385,0 +11386,0 +11387,0 +11388,0 +11389,0 +11390,0 +11391,0 +11392,0 +11393,0 +11394,0 +11395,0 +11396,0 +11397,0 +11398,0 +11399,0 +11400,0 +11401,0 +11402,0 +11403,0 +11404,0 +11405,0 +11406,0 +11407,0 +11408,0 +11409,0 +11410,0 +11411,0 +11412,0 +11413,0 +11414,0 +11415,0 +11416,0 +11417,0 +11418,0 +11419,0 +11420,0 +11421,0 +11422,0 +11423,0 +11424,0 +11425,0 +11426,0 +11427,0 +11428,0 +11429,0 +11430,0 +11431,0 +11432,0 +11433,0 +11434,0 +11435,0 +11436,0 +11437,0 +11438,0 +11439,0 +11440,0 +11441,0 +11442,0 +11443,0 +11444,0 +11445,0 +11446,0 +11447,0 +11448,0 +11449,0 +11450,0 +11451,0 +11452,0 +11453,0 +11454,0 +11455,0 +11456,0 +11457,0 +11458,0 +11459,0 +11460,0 +11461,0 +11462,0 +11463,0 +11464,0 +11465,0 +11466,0 +11467,0 +11468,0 +11469,0 +11470,0 +11471,0 +11472,0 +11473,0 +11474,0 +11475,0 +11476,0 +11477,0 +11478,0 +11479,0 +11480,0 +11481,0 +11482,0 +11483,0 +11484,0 +11485,0 +11486,0 +11487,0 +11488,0 +11489,0 +11490,0 +11491,0 +11492,0 +11493,0 +11494,0 +11495,0 +11496,0 +11497,0 +11498,0 +11499,0 +11500,0 +11501,0 +11502,0 +11503,0 +11504,0 +11505,0 +11506,0 +11507,0 +11508,0 +11509,0 +11510,0 +11511,0 +11512,0 +11513,0 +11514,0 +11515,0 +11516,0 +11517,0 +11518,0 +11519,0 +11520,0 +11521,0 +11522,0 +11523,0 +11524,0 +11525,0 +11526,0 +11527,0 +11528,0 +11529,0 +11530,0 +11531,0 +11532,0 +11533,0 +11534,0 +11535,0 +11536,0 +11537,0 +11538,0 +11539,0 +11540,0 +11541,0 +11542,0 +11543,0 +11544,0 +11545,0 +11546,0 +11547,0 +11548,0 +11549,0 +11550,0 +11551,0 +11552,0 +11553,0 +11554,0 +11555,0 +11556,0 +11557,0 +11558,0 +11559,0 +11560,0 +11561,0 +11562,0 +11563,0 +11564,0 +11565,0 +11566,0 +11567,0 +11568,0 +11569,0 +11570,0 +11571,0 +11572,0 +11573,0 +11574,0 +11575,0 +11576,0 +11577,0 +11578,0 +11579,0 +11580,0 +11581,0 +11582,0 +11583,0 +11584,0 +11585,0 +11586,0 +11587,0 +11588,0 +11589,0 +11590,0 +11591,0 +11592,0 +11593,0 +11594,0 +11595,0 +11596,0 +11597,0 +11598,0 +11599,0 +11600,0 +11601,0 +11602,0 +11603,0 +11604,0 +11605,0 +11606,0 +11607,0 +11608,0 +11609,0 +11610,0 +11611,0 +11612,0 +11613,0 +11614,0 +11615,0 +11616,0 +11617,0 +11618,0 +11619,0 +11620,0 +11621,0 +11622,0 +11623,0 +11624,0 +11625,0 +11626,0 +11627,0 +11628,0 +11629,0 +11630,0 +11631,0 +11632,0 +11633,0 +11634,0 +11635,0 +11636,0 +11637,0 +11638,0 +11639,0 +11640,0 +11641,0 +11642,0 +11643,0 +11644,0 +11645,0 +11646,0 +11647,0 +11648,0 +11649,0 +11650,0 +11651,0 +11652,0 +11653,0 +11654,0 +11655,0 +11656,0 +11657,0 +11658,0 +11659,0 +11660,0 +11661,0 +11662,0 +11663,0 +11664,0 +11665,0 +11666,0 +11667,0 +11668,0 +11669,0 +11670,0 +11671,0 +11672,0 +11673,0 +11674,0 +11675,0 +11676,0 +11677,0 +11678,0 +11679,0 +11680,0 +11681,0 +11682,0 +11683,0 +11684,0 +11685,0 +11686,0 +11687,0 +11688,0 +11689,0 +11690,0 +11691,0 +11692,0 +11693,0 +11694,0 +11695,0 +11696,0 +11697,0 +11698,0 +11699,0 +11700,0 +11701,0 +11702,0 +11703,0 +11704,0 +11705,0 +11706,0 +11707,0 +11708,0 +11709,0 +11710,0 +11711,0 +11712,0 +11713,0 +11714,0 +11715,0 +11716,0 +11717,0 +11718,0 +11719,0 +11720,0 +11721,0 +11722,0 +11723,0 +11724,0 +11725,0 +11726,0 +11727,0 +11728,0 +11729,0 +11730,0 +11731,0 +11732,0 +11733,0 +11734,0 +11735,0 +11736,0 +11737,0 +11738,0 +11739,0 +11740,0 +11741,0 +11742,0 +11743,0 +11744,0 +11745,0 +11746,0 +11747,0 +11748,0 +11749,0 +11750,0 +11751,0 +11752,0 +11753,0 +11754,0 +11755,0 +11756,0 +11757,0 +11758,0 +11759,0 +11760,0 +11761,0 +11762,0 +11763,0 +11764,0 +11765,0 +11766,0 +11767,0 +11768,0 +11769,0 +11770,0 +11771,0 +11772,0 +11773,0 +11774,0 +11775,0 +11776,0 +11777,0 +11778,0 +11779,0 +11780,0 +11781,0 +11782,0 +11783,0 +11784,0 +11785,0 +11786,0 +11787,0 +11788,0 +11789,0 +11790,0 +11791,0 +11792,0 +11793,0 +11794,0 +11795,0 +11796,0 +11797,0 +11798,0 +11799,0 +11800,0 +11801,0 +11802,0 +11803,0 +11804,0 +11805,0 +11806,0 +11807,0 +11808,0 +11809,0 +11810,0 +11811,0 +11812,0 +11813,0 +11814,0 +11815,0 +11816,0 +11817,0 +11818,0 +11819,0 +11820,0 +11821,0 +11822,0 +11823,0 +11824,0 +11825,0 +11826,0 +11827,0 +11828,0 +11829,0 +11830,0 +11831,0 +11832,0 +11833,0 +11834,0 +11835,0 +11836,0 +11837,0 +11838,0 +11839,0 +11840,0 +11841,0 +11842,0 +11843,0 +11844,0 +11845,0 +11846,0 +11847,0 +11848,0 +11849,0 +11850,0 +11851,0 +11852,0 +11853,0 +11854,0 +11855,0 +11856,0 +11857,0 +11858,0 +11859,0 +11860,0 +11861,0 +11862,0 +11863,0 +11864,0 +11865,0 +11866,0 +11867,0 +11868,0 +11869,0 +11870,0 +11871,0 +11872,0 +11873,0 +11874,0 +11875,0 +11876,0 +11877,0 +11878,0 +11879,0 +11880,0 +11881,0 +11882,0 +11883,0 +11884,0 +11885,0 +11886,0 +11887,0 +11888,0 +11889,0 +11890,0 +11891,0 +11892,0 +11893,0 +11894,0 +11895,0 +11896,0 +11897,0 +11898,0 +11899,0 +11900,0 +11901,0 +11902,0 +11903,0 +11904,0 +11905,0 +11906,0 +11907,0 +11908,0 +11909,0 +11910,0 +11911,0 +11912,0 +11913,0 +11914,0 +11915,0 +11916,0 +11917,0 +11918,0 +11919,0 +11920,0 +11921,0 +11922,0 +11923,0 +11924,0 +11925,0 +11926,0 +11927,0 +11928,0 +11929,0 +11930,0 +11931,0 +11932,0 +11933,0 +11934,0 +11935,0 +11936,0 +11937,0 +11938,0 +11939,0 +11940,0 +11941,0 +11942,0 +11943,0 +11944,0 +11945,0 +11946,0 +11947,0 +11948,0 +11949,0 +11950,0 +11951,0 +11952,0 +11953,0 +11954,0 +11955,0 +11956,0 +11957,0 +11958,0 +11959,0 +11960,0 +11961,0 +11962,0 +11963,0 +11964,0 +11965,0 +11966,0 +11967,0 +11968,0 +11969,0 +11970,0 +11971,0 +11972,0 +11973,0 +11974,0 +11975,0 +11976,0 +11977,0 +11978,0 +11979,0 +11980,0 +11981,0 +11982,0 +11983,0 +11984,0 +11985,0 +11986,0 +11987,0 +11988,0 +11989,0 +11990,0 +11991,0 +11992,0 +11993,0 +11994,0 +11995,0 +11996,0 +11997,0 +11998,0 +11999,0 +12000,0 +12001,0 +12002,0 +12003,0 +12004,0 +12005,0 +12006,0 +12007,0 +12008,0 +12009,0 +12010,0 +12011,0 +12012,0 +12013,0 +12014,0 +12015,0 +12016,0 +12017,0 +12018,0 +12019,0 +12020,0 +12021,0 +12022,0 +12023,0 +12024,0 +12025,0 +12026,0 +12027,0 +12028,0 +12029,0 +12030,0 +12031,0 +12032,0 +12033,0 +12034,0 +12035,0 +12036,0 +12037,0 +12038,0 +12039,0 +12040,0 +12041,0 +12042,0 +12043,0 +12044,0 +12045,0 +12046,0 +12047,0 +12048,0 +12049,0 +12050,0 +12051,0 +12052,0 +12053,0 +12054,0 +12055,0 +12056,0 +12057,0 +12058,0 +12059,0 +12060,0 +12061,0 +12062,0 +12063,0 +12064,0 +12065,0 +12066,0 +12067,0 +12068,0 +12069,0 +12070,0 +12071,0 +12072,0 +12073,0 +12074,0 +12075,0 +12076,0 +12077,0 +12078,0 +12079,0 +12080,0 +12081,0 +12082,0 +12083,0 +12084,0 +12085,0 +12086,0 +12087,0 +12088,0 +12089,0 +12090,0 +12091,0 +12092,0 +12093,0 +12094,0 +12095,0 +12096,0 +12097,0 +12098,0 +12099,0 +12100,0 +12101,0 +12102,0 +12103,0 +12104,0 +12105,0 +12106,0 +12107,0 +12108,0 +12109,0 +12110,0 +12111,0 +12112,0 +12113,0 +12114,0 +12115,0 +12116,0 +12117,0 +12118,0 +12119,0 +12120,0 +12121,0 +12122,0 +12123,0 +12124,0 +12125,0 +12126,0 +12127,0 +12128,0 +12129,0 +12130,0 +12131,0 +12132,0 +12133,0 +12134,0 +12135,0 +12136,0 +12137,0 +12138,0 +12139,0 +12140,0 +12141,0 +12142,0 +12143,0 +12144,0 +12145,0 +12146,0 +12147,0 +12148,0 +12149,0 +12150,0 +12151,0 +12152,0 +12153,0 +12154,0 +12155,0 +12156,0 +12157,0 +12158,0 +12159,0 +12160,0 +12161,0 +12162,0 +12163,0 +12164,0 +12165,0 +12166,0 +12167,0 +12168,0 +12169,0 +12170,0 +12171,0 +12172,0 +12173,0 +12174,0 +12175,0 +12176,0 +12177,0 +12178,0 +12179,0 +12180,0 +12181,0 +12182,0 +12183,0 +12184,0 +12185,0 +12186,0 +12187,0 +12188,0 +12189,0 +12190,0 +12191,0 +12192,0 +12193,0 +12194,0 +12195,0 +12196,0 +12197,0 +12198,0 +12199,0 +12200,0 +12201,0 +12202,0 +12203,0 +12204,0 +12205,0 +12206,0 +12207,0 +12208,0 +12209,0 +12210,0 +12211,0 +12212,0 +12213,0 +12214,0 +12215,0 +12216,0 +12217,0 +12218,0 +12219,0 +12220,0 +12221,0 +12222,0 +12223,0 +12224,0 +12225,0 +12226,0 +12227,0 +12228,0 +12229,0 +12230,0 +12231,0 +12232,0 +12233,0 +12234,0 +12235,0 +12236,0 +12237,0 +12238,0 +12239,0 +12240,0 +12241,0 +12242,0 +12243,0 +12244,0 +12245,0 +12246,0 +12247,0 +12248,0 +12249,0 +12250,0 +12251,0 +12252,0 +12253,0 +12254,0 +12255,0 +12256,0 +12257,0 +12258,0 +12259,0 +12260,0 +12261,0 +12262,0 +12263,0 +12264,0 +12265,0 +12266,0 +12267,0 +12268,0 +12269,0 +12270,0 +12271,0 +12272,0 +12273,0 +12274,0 +12275,0 +12276,0 +12277,0 +12278,0 +12279,0 +12280,0 +12281,0 +12282,0 +12283,0 +12284,0 +12285,0 +12286,0 +12287,0 +12288,0 +12289,0 +12290,0 +12291,0 +12292,0 +12293,0 +12294,0 +12295,0 +12296,0 +12297,0 +12298,0 +12299,0 +12300,0 +12301,0 +12302,0 +12303,0 +12304,0 +12305,0 +12306,0 +12307,0 +12308,0 +12309,0 +12310,0 +12311,0 +12312,0 +12313,0 +12314,0 +12315,0 +12316,0 +12317,0 +12318,0 +12319,0 +12320,0 +12321,0 +12322,0 +12323,0 +12324,0 +12325,0 +12326,0 +12327,0 +12328,0 +12329,0 +12330,0 +12331,0 +12332,0 +12333,0 +12334,0 +12335,0 +12336,0 +12337,0 +12338,0 +12339,0 +12340,0 +12341,0 +12342,0 +12343,0 +12344,0 +12345,0 +12346,0 +12347,0 +12348,0 +12349,0 +12350,0 +12351,0 +12352,0 +12353,0 +12354,0 +12355,0 +12356,0 +12357,0 +12358,0 +12359,0 +12360,0 +12361,0 +12362,0 +12363,0 +12364,0 +12365,0 +12366,0 +12367,0 +12368,0 +12369,0 +12370,0 +12371,0 +12372,0 +12373,0 +12374,0 +12375,0 +12376,0 +12377,0 +12378,0 +12379,0 +12380,0 +12381,0 +12382,0 +12383,0 +12384,0 +12385,0 +12386,0 +12387,0 +12388,0 +12389,0 +12390,0 +12391,0 +12392,0 +12393,0 +12394,0 +12395,0 +12396,0 +12397,0 +12398,0 +12399,0 +12400,0 +12401,0 +12402,0 +12403,0 +12404,0 +12405,0 +12406,0 +12407,0 +12408,0 +12409,0 +12410,0 +12411,0 +12412,0 +12413,0 +12414,0 +12415,0 +12416,0 +12417,0 +12418,0 +12419,0 +12420,0 +12421,0 +12422,0 +12423,0 +12424,0 +12425,0 +12426,0 +12427,0 +12428,0 +12429,0 +12430,0 +12431,0 +12432,0 +12433,0 +12434,0 +12435,0 +12436,0 +12437,0 +12438,0 +12439,0 +12440,0 +12441,0 +12442,0 +12443,0 +12444,0 +12445,0 +12446,0 +12447,0 +12448,0 +12449,0 +12450,0 +12451,0 +12452,0 +12453,0 +12454,0 +12455,0 +12456,0 +12457,0 +12458,0 +12459,0 +12460,0 +12461,0 +12462,0 +12463,0 +12464,0 +12465,0 +12466,0 +12467,0 +12468,0 +12469,0 +12470,0 +12471,0 +12472,0 +12473,0 +12474,0 +12475,0 +12476,0 +12477,0 +12478,0 +12479,0 +12480,0 +12481,0 +12482,0 +12483,0 +12484,0 +12485,0 +12486,0 +12487,0 +12488,0 +12489,0 +12490,0 +12491,0 +12492,0 +12493,0 +12494,0 +12495,0 +12496,0 +12497,0 +12498,0 +12499,0 +12500,0 +12501,0 +12502,0 +12503,0 +12504,0 +12505,0 +12506,0 +12507,0 +12508,0 +12509,0 +12510,0 +12511,0 +12512,0 +12513,0 +12514,0 +12515,0 +12516,0 +12517,0 +12518,0 +12519,0 +12520,0 +12521,0 +12522,0 +12523,0 +12524,0 +12525,0 +12526,0 +12527,0 +12528,0 +12529,0 +12530,0 +12531,0 +12532,0 +12533,0 +12534,0 +12535,0 +12536,0 +12537,0 +12538,0 +12539,0 +12540,0 +12541,0 +12542,0 +12543,0 +12544,0 +12545,0 +12546,0 +12547,0 +12548,0 +12549,0 +12550,0 +12551,0 +12552,0 +12553,0 +12554,0 +12555,0 +12556,0 +12557,0 +12558,0 +12559,0 +12560,0 +12561,0 +12562,0 +12563,0 +12564,0 +12565,0 +12566,0 +12567,0 +12568,0 +12569,0 +12570,0 +12571,0 +12572,0 +12573,0 +12574,0 +12575,0 +12576,0 +12577,0 +12578,0 +12579,0 +12580,0 +12581,0 +12582,0 +12583,0 +12584,0 +12585,0 +12586,0 +12587,0 +12588,0 +12589,0 +12590,0 +12591,0 +12592,0 +12593,0 +12594,0 +12595,0 +12596,0 +12597,0 +12598,0 +12599,0 +12600,0 +12601,0 +12602,0 +12603,0 +12604,0 +12605,0 +12606,0 +12607,0 +12608,0 +12609,0 +12610,0 +12611,0 +12612,0 +12613,0 +12614,0 +12615,0 +12616,0 +12617,0 +12618,0 +12619,0 +12620,0 +12621,0 +12622,0 +12623,0 +12624,0 +12625,0 +12626,0 +12627,0 +12628,0 +12629,0 +12630,0 +12631,0 +12632,0 +12633,0 +12634,0 +12635,0 +12636,0 +12637,0 +12638,0 +12639,0 +12640,0 +12641,0 +12642,0 +12643,0 +12644,0 +12645,0 +12646,0 +12647,0 +12648,0 +12649,0 +12650,0 +12651,0 +12652,0 +12653,0 +12654,0 +12655,0 +12656,0 +12657,0 +12658,0 +12659,0 +12660,0 +12661,0 +12662,0 +12663,0 +12664,0 +12665,0 +12666,0 +12667,0 +12668,0 +12669,0 +12670,0 +12671,0 +12672,0 +12673,0 +12674,0 +12675,0 +12676,0 +12677,0 +12678,0 +12679,0 +12680,0 +12681,0 +12682,0 +12683,0 +12684,0 +12685,0 +12686,0 +12687,0 +12688,0 +12689,0 +12690,0 +12691,0 +12692,0 +12693,0 +12694,0 +12695,0 +12696,0 +12697,0 +12698,0 +12699,0 +12700,0 +12701,0 +12702,0 +12703,0 +12704,0 +12705,0 +12706,0 +12707,0 +12708,0 +12709,0 +12710,0 +12711,0 +12712,0 +12713,0 +12714,0 +12715,0 +12716,0 +12717,0 +12718,0 +12719,0 +12720,0 +12721,0 +12722,0 +12723,0 +12724,0 +12725,0 +12726,0 +12727,0 +12728,0 +12729,0 +12730,0 +12731,0 +12732,0 +12733,0 +12734,0 +12735,0 +12736,0 +12737,0 +12738,0 +12739,0 +12740,0 +12741,0 +12742,0 +12743,0 +12744,0 +12745,0 +12746,0 +12747,0 +12748,0 +12749,0 +12750,0 +12751,0 +12752,0 +12753,0 +12754,0 +12755,0 +12756,0 +12757,0 +12758,0 +12759,0 +12760,0 +12761,0 +12762,0 +12763,0 +12764,0 +12765,0 +12766,0 +12767,0 +12768,0 +12769,0 +12770,0 +12771,0 +12772,0 +12773,0 +12774,0 +12775,0 +12776,0 +12777,0 +12778,0 +12779,0 +12780,0 +12781,0 +12782,0 +12783,0 +12784,0 +12785,0 +12786,0 +12787,0 +12788,0 +12789,0 +12790,0 +12791,0 +12792,0 +12793,0 +12794,0 +12795,0 +12796,0 +12797,0 +12798,0 +12799,0 +12800,0 +12801,0 +12802,0 +12803,0 +12804,0 +12805,0 +12806,0 +12807,0 +12808,0 +12809,0 +12810,0 +12811,0 +12812,0 +12813,0 +12814,0 +12815,0 +12816,0 +12817,0 +12818,0 +12819,0 +12820,0 +12821,0 +12822,0 +12823,0 +12824,0 +12825,0 +12826,0 +12827,0 +12828,0 +12829,0 +12830,0 +12831,0 +12832,0 +12833,0 +12834,0 +12835,0 +12836,0 +12837,0 +12838,0 +12839,0 +12840,0 +12841,0 +12842,0 +12843,0 +12844,0 +12845,0 +12846,0 +12847,0 +12848,0 +12849,0 +12850,0 +12851,0 +12852,0 +12853,0 +12854,0 +12855,0 +12856,0 +12857,0 +12858,0 +12859,0 +12860,0 +12861,0 +12862,0 +12863,0 +12864,0 +12865,0 +12866,0 +12867,0 +12868,0 +12869,0 +12870,0 +12871,0 +12872,0 +12873,0 +12874,0 +12875,0 +12876,0 +12877,0 +12878,0 +12879,0 +12880,0 +12881,0 +12882,0 +12883,0 +12884,0 +12885,0 +12886,0 +12887,0 +12888,0 +12889,0 +12890,0 +12891,0 +12892,0 +12893,0 +12894,0 +12895,0 +12896,0 +12897,0 +12898,0 +12899,0 +12900,0 +12901,0 +12902,0 +12903,0 +12904,0 +12905,0 +12906,0 +12907,0 +12908,0 +12909,0 +12910,0 +12911,0 +12912,0 +12913,0 +12914,0 +12915,0 +12916,0 +12917,0 +12918,0 +12919,0 +12920,0 +12921,0 +12922,0 +12923,0 +12924,0 +12925,0 +12926,0 +12927,0 +12928,0 +12929,0 +12930,0 +12931,0 +12932,0 +12933,0 +12934,0 +12935,0 +12936,0 +12937,0 +12938,0 +12939,0 +12940,0 +12941,0 +12942,0 +12943,0 +12944,0 +12945,0 +12946,0 +12947,0 +12948,0 +12949,0 +12950,0 +12951,0 +12952,0 +12953,0 +12954,0 +12955,0 +12956,0 +12957,0 +12958,0 +12959,0 +12960,0 +12961,0 +12962,0 +12963,0 +12964,0 +12965,0 +12966,0 +12967,0 +12968,0 +12969,0 +12970,0 +12971,0 +12972,0 +12973,0 +12974,0 +12975,0 +12976,0 +12977,0 +12978,0 +12979,0 +12980,0 +12981,0 +12982,0 +12983,0 +12984,0 +12985,0 +12986,0 +12987,0 +12988,0 +12989,0 +12990,0 +12991,0 +12992,0 +12993,0 +12994,0 +12995,0 +12996,0 +12997,0 +12998,0 +12999,0 +13000,0 +13001,0 +13002,0 +13003,0 +13004,0 +13005,0 +13006,0 +13007,0 +13008,0 +13009,0 +13010,0 +13011,0 +13012,0 +13013,0 +13014,0 +13015,0 +13016,0 +13017,0 +13018,0 +13019,0 +13020,0 +13021,0 +13022,0 +13023,0 +13024,0 +13025,0 +13026,0 +13027,0 +13028,0 +13029,0 +13030,0 +13031,0 +13032,0 +13033,0 +13034,0 +13035,0 +13036,0 +13037,0 +13038,0 +13039,0 +13040,0 +13041,0 +13042,0 +13043,0 +13044,0 +13045,0 +13046,0 +13047,0 +13048,0 +13049,0 +13050,0 +13051,0 +13052,0 +13053,0 +13054,0 +13055,0 +13056,0 +13057,0 +13058,0 +13059,0 +13060,0 +13061,0 +13062,0 +13063,0 +13064,0 +13065,0 +13066,0 +13067,0 +13068,0 +13069,0 +13070,0 +13071,0 +13072,0 +13073,0 +13074,0 +13075,0 +13076,0 +13077,0 +13078,0 +13079,0 +13080,0 +13081,0 +13082,0 +13083,0 +13084,0 +13085,0 +13086,0 +13087,0 +13088,0 +13089,0 +13090,0 +13091,0 +13092,0 +13093,0 +13094,0 +13095,0 +13096,0 +13097,0 +13098,0 +13099,0 +13100,0 +13101,0 +13102,0 +13103,0 +13104,0 +13105,0 +13106,0 +13107,0 +13108,0 +13109,0 +13110,0 +13111,0 +13112,0 +13113,0 +13114,0 +13115,0 +13116,0 +13117,0 +13118,0 +13119,0 +13120,0 +13121,0 +13122,0 +13123,0 +13124,0 +13125,0 +13126,0 +13127,0 +13128,0 +13129,0 +13130,0 +13131,0 +13132,0 +13133,0 +13134,0 +13135,0 +13136,0 +13137,0 +13138,0 +13139,0 +13140,0 +13141,0 +13142,0 +13143,0 +13144,0 +13145,0 +13146,0 +13147,0 +13148,0 +13149,0 +13150,0 +13151,0 +13152,0 +13153,0 +13154,0 +13155,0 +13156,0 +13157,0 +13158,0 +13159,0 +13160,0 +13161,0 +13162,0 +13163,0 +13164,0 +13165,0 +13166,0 +13167,0 +13168,0 +13169,0 +13170,0 +13171,0 +13172,0 +13173,0 +13174,0 +13175,0 +13176,0 +13177,0 +13178,0 +13179,0 +13180,0 +13181,0 +13182,0 +13183,0 +13184,0 +13185,0 +13186,0 +13187,0 +13188,0 +13189,0 +13190,0 +13191,0 +13192,0 +13193,0 +13194,0 +13195,0 +13196,0 +13197,0 +13198,0 +13199,0 +13200,0 +13201,0 +13202,0 +13203,0 +13204,0 +13205,0 +13206,0 +13207,0 +13208,0 +13209,0 +13210,0 +13211,0 +13212,0 +13213,0 +13214,0 +13215,0 +13216,0 +13217,0 +13218,0 +13219,0 +13220,0 +13221,0 +13222,0 +13223,0 +13224,0 +13225,0 +13226,0 +13227,0 +13228,0 +13229,0 +13230,0 +13231,0 +13232,0 +13233,0 +13234,0 +13235,0 +13236,0 +13237,0 +13238,0 +13239,0 +13240,0 +13241,0 +13242,0 +13243,0 +13244,0 +13245,0 +13246,0 +13247,0 +13248,0 +13249,0 +13250,0 +13251,0 +13252,0 +13253,0 +13254,0 +13255,0 +13256,0 +13257,0 +13258,0 +13259,0 +13260,0 +13261,0 +13262,0 +13263,0 +13264,0 +13265,0 +13266,0 +13267,0 +13268,0 +13269,0 +13270,0 +13271,0 +13272,0 +13273,0 +13274,0 +13275,0 +13276,0 +13277,0 +13278,0 +13279,0 +13280,0 +13281,0 +13282,0 +13283,0 +13284,0 +13285,0 +13286,0 +13287,0 +13288,0 +13289,0 +13290,0 +13291,0 +13292,0 +13293,0 +13294,0 +13295,0 +13296,0 +13297,0 +13298,0 +13299,0 +13300,0 +13301,0 +13302,0 +13303,0 +13304,0 +13305,0 +13306,0 +13307,0 +13308,0 +13309,0 +13310,0 +13311,0 +13312,0 +13313,0 +13314,0 +13315,0 +13316,0 +13317,0 +13318,0 +13319,0 +13320,0 +13321,0 +13322,0 +13323,0 +13324,0 +13325,0 +13326,0 +13327,0 +13328,0 +13329,0 +13330,0 +13331,0 +13332,0 +13333,0 +13334,0 +13335,0 +13336,0 +13337,0 +13338,0 +13339,0 +13340,0 +13341,0 +13342,0 +13343,0 +13344,0 +13345,0 +13346,0 +13347,0 +13348,0 +13349,0 +13350,0 +13351,0 +13352,0 +13353,0 +13354,0 +13355,0 +13356,0 +13357,0 +13358,0 +13359,0 +13360,0 +13361,0 +13362,0 +13363,0 +13364,0 +13365,0 +13366,0 +13367,0 +13368,0 +13369,0 +13370,0 +13371,0 +13372,0 +13373,0 +13374,0 +13375,0 +13376,0 +13377,0 +13378,0 +13379,0 +13380,0 +13381,0 +13382,0 +13383,0 +13384,0 +13385,0 +13386,0 +13387,0 +13388,0 +13389,0 +13390,0 +13391,0 +13392,0 +13393,0 +13394,0 +13395,0 +13396,0 +13397,0 +13398,0 +13399,0 +13400,0 +13401,0 +13402,0 +13403,0 +13404,0 +13405,0 +13406,0 +13407,0 +13408,0 +13409,0 +13410,0 +13411,0 +13412,0 +13413,0 +13414,0 +13415,0 +13416,0 +13417,0 +13418,0 +13419,0 +13420,0 +13421,0 +13422,0 +13423,0 +13424,0 +13425,0 +13426,0 +13427,0 +13428,0 +13429,0 +13430,0 +13431,0 +13432,0 +13433,0 +13434,0 +13435,0 +13436,0 +13437,0 +13438,0 +13439,0 +13440,0 +13441,0 +13442,0 +13443,0 +13444,0 +13445,0 +13446,0 +13447,0 +13448,0 +13449,0 +13450,0 +13451,0 +13452,0 +13453,0 +13454,0 +13455,0 +13456,0 +13457,0 +13458,0 +13459,0 +13460,0 +13461,0 +13462,0 +13463,0 +13464,0 +13465,0 +13466,0 +13467,0 +13468,0 +13469,0 +13470,0 +13471,0 +13472,0 +13473,0 +13474,0 +13475,0 +13476,0 +13477,0 +13478,0 +13479,0 +13480,0 +13481,0 +13482,0 +13483,0 +13484,0 +13485,0 +13486,0 +13487,0 +13488,0 +13489,0 +13490,0 +13491,0 +13492,0 +13493,0 +13494,0 +13495,0 +13496,0 +13497,0 +13498,0 +13499,0 +13500,0 +13501,0 +13502,0 +13503,0 +13504,0 +13505,0 +13506,0 +13507,0 +13508,0 +13509,0 +13510,0 +13511,0 +13512,0 +13513,0 +13514,0 +13515,0 +13516,0 +13517,0 +13518,0 +13519,0 +13520,0 +13521,0 +13522,0 +13523,0 +13524,0 +13525,0 +13526,0 +13527,0 +13528,0 +13529,0 +13530,0 +13531,0 +13532,0 +13533,0 +13534,0 +13535,0 +13536,0 +13537,0 +13538,0 +13539,0 +13540,0 +13541,0 +13542,0 +13543,0 +13544,0 +13545,0 +13546,0 +13547,0 +13548,0 +13549,0 +13550,0 +13551,0 +13552,0 +13553,0 +13554,0 +13555,0 +13556,0 +13557,0 +13558,0 +13559,0 +13560,0 +13561,0 +13562,0 +13563,0 +13564,0 +13565,0 +13566,0 +13567,0 +13568,0 +13569,0 +13570,0 +13571,0 +13572,0 +13573,0 +13574,0 +13575,0 +13576,0 +13577,0 +13578,0 +13579,0 +13580,0 +13581,0 +13582,0 +13583,0 +13584,0 +13585,0 +13586,0 +13587,0 +13588,0 +13589,0 +13590,0 +13591,0 +13592,0 +13593,0 +13594,0 +13595,0 +13596,0 +13597,0 +13598,0 +13599,0 +13600,0 +13601,0 +13602,0 +13603,0 +13604,0 +13605,0 +13606,0 +13607,0 +13608,0 +13609,0 +13610,0 +13611,0 +13612,0 +13613,0 +13614,0 +13615,0 +13616,0 +13617,0 +13618,0 +13619,0 +13620,0 +13621,0 +13622,0 +13623,0 +13624,0 +13625,0 +13626,0 +13627,0 +13628,0 +13629,0 +13630,0 +13631,0 +13632,0 +13633,0 +13634,0 +13635,0 +13636,0 +13637,0 +13638,0 +13639,0 +13640,0 +13641,0 +13642,0 +13643,0 +13644,0 +13645,0 +13646,0 +13647,0 +13648,0 +13649,0 +13650,0 +13651,0 +13652,0 +13653,0 +13654,0 +13655,0 +13656,0 +13657,0 +13658,0 +13659,0 +13660,0 +13661,0 +13662,0 +13663,0 +13664,0 +13665,0 +13666,0 +13667,0 +13668,0 +13669,0 +13670,0 +13671,0 +13672,0 +13673,0 +13674,0 +13675,0 +13676,0 +13677,0 +13678,0 +13679,0 +13680,0 +13681,0 +13682,0 +13683,0 +13684,0 +13685,0 +13686,0 +13687,0 +13688,0 +13689,0 +13690,0 +13691,0 +13692,0 +13693,0 +13694,0 +13695,0 +13696,0 +13697,0 +13698,0 +13699,0 +13700,0 +13701,0 +13702,0 +13703,0 +13704,0 +13705,0 +13706,0 +13707,0 +13708,0 +13709,0 +13710,0 +13711,0 +13712,0 +13713,0 +13714,0 +13715,0 +13716,0 +13717,0 +13718,0 +13719,0 +13720,0 +13721,0 +13722,0 +13723,0 +13724,0 +13725,0 +13726,0 +13727,0 +13728,0 +13729,0 +13730,0 +13731,0 +13732,0 +13733,0 +13734,0 +13735,0 +13736,0 +13737,0 +13738,0 +13739,0 +13740,0 +13741,0 +13742,0 +13743,0 +13744,0 +13745,0 +13746,0 +13747,0 +13748,0 +13749,0 +13750,0 +13751,0 +13752,0 +13753,0 +13754,0 +13755,0 +13756,0 +13757,0 +13758,0 +13759,0 +13760,0 +13761,0 +13762,0 +13763,0 +13764,0 +13765,0 +13766,0 +13767,0 +13768,0 +13769,0 +13770,0 +13771,0 +13772,0 +13773,0 +13774,0 +13775,0 +13776,0 +13777,0 +13778,0 +13779,0 +13780,0 +13781,0 +13782,0 +13783,0 +13784,0 +13785,0 +13786,0 +13787,0 +13788,0 +13789,0 +13790,0 +13791,0 +13792,0 +13793,0 +13794,0 +13795,0 +13796,0 +13797,0 +13798,0 +13799,0 +13800,0 +13801,0 +13802,0 +13803,0 +13804,0 +13805,0 +13806,0 +13807,0 +13808,0 +13809,0 +13810,0 +13811,0 +13812,0 +13813,0 +13814,0 +13815,0 +13816,0 +13817,0 +13818,0 +13819,0 +13820,0 +13821,0 +13822,0 +13823,0 +13824,0 +13825,0 +13826,0 +13827,0 +13828,0 +13829,0 +13830,0 +13831,0 +13832,0 +13833,0 +13834,0 +13835,0 +13836,0 +13837,0 +13838,0 +13839,0 +13840,0 +13841,0 +13842,0 +13843,0 +13844,0 +13845,0 +13846,0 +13847,0 +13848,0 +13849,0 +13850,0 +13851,0 +13852,0 +13853,0 +13854,0 +13855,0 +13856,0 +13857,0 +13858,0 +13859,0 +13860,0 +13861,0 +13862,0 +13863,0 +13864,0 +13865,0 +13866,0 +13867,0 +13868,0 +13869,0 +13870,0 +13871,0 +13872,0 +13873,0 +13874,0 +13875,0 +13876,0 +13877,0 +13878,0 +13879,0 +13880,0 +13881,0 +13882,0 +13883,0 +13884,0 +13885,0 +13886,0 +13887,0 +13888,0 +13889,0 +13890,0 +13891,0 +13892,0 +13893,0 +13894,0 +13895,0 +13896,0 +13897,0 +13898,0 +13899,0 +13900,0 +13901,0 +13902,0 +13903,0 +13904,0 +13905,0 +13906,0 +13907,0 +13908,0 +13909,0 +13910,0 +13911,0 +13912,0 +13913,0 +13914,0 +13915,0 +13916,0 +13917,0 +13918,0 +13919,0 +13920,0 +13921,0 +13922,0 +13923,0 +13924,0 +13925,0 +13926,0 +13927,0 +13928,0 +13929,0 +13930,0 +13931,0 +13932,0 +13933,0 +13934,0 +13935,0 +13936,0 +13937,0 +13938,0 +13939,0 +13940,0 +13941,0 +13942,0 +13943,0 +13944,0 +13945,0 +13946,0 +13947,0 +13948,0 +13949,0 +13950,0 +13951,0 +13952,0 +13953,0 +13954,0 +13955,0 +13956,0 +13957,0 +13958,0 +13959,0 +13960,0 +13961,0 +13962,0 +13963,0 +13964,0 +13965,0 +13966,0 +13967,0 +13968,0 +13969,0 +13970,0 +13971,0 +13972,0 +13973,0 +13974,0 +13975,0 +13976,0 +13977,0 +13978,0 +13979,0 +13980,0 +13981,0 +13982,0 +13983,0 +13984,0 +13985,0 +13986,0 +13987,0 +13988,0 +13989,0 +13990,0 +13991,0 +13992,0 +13993,0 +13994,0 +13995,0 +13996,0 +13997,0 +13998,0 +13999,0 +14000,0 +14001,0 +14002,0 +14003,0 +14004,0 +14005,0 +14006,0 +14007,0 +14008,0 +14009,0 +14010,0 +14011,0 +14012,0 +14013,0 +14014,0 +14015,0 +14016,0 +14017,0 +14018,0 +14019,0 +14020,0 +14021,0 +14022,0 +14023,0 +14024,0 +14025,0 +14026,0 +14027,0 +14028,0 +14029,0 +14030,0 +14031,0 +14032,0 +14033,0 +14034,0 +14035,0 +14036,0 +14037,0 +14038,0 +14039,0 +14040,0 +14041,0 +14042,0 +14043,0 +14044,0 +14045,0 +14046,0 +14047,0 +14048,0 +14049,0 +14050,0 +14051,0 +14052,0 +14053,0 +14054,0 +14055,0 +14056,0 +14057,0 +14058,0 +14059,0 +14060,0 +14061,0 +14062,0 +14063,0 +14064,0 +14065,0 +14066,0 +14067,0 +14068,0 +14069,0 +14070,0 +14071,0 +14072,0 +14073,0 +14074,0 +14075,0 +14076,0 +14077,0 +14078,0 +14079,0 +14080,0 +14081,0 +14082,0 +14083,0 +14084,0 +14085,0 +14086,0 +14087,0 +14088,0 +14089,0 +14090,0 +14091,0 +14092,0 +14093,0 +14094,0 +14095,0 +14096,0 +14097,0 +14098,0 +14099,0 +14100,0 +14101,0 +14102,0 +14103,0 +14104,0 +14105,0 +14106,0 +14107,0 +14108,0 +14109,0 +14110,0 +14111,0 +14112,0 +14113,0 +14114,0 +14115,0 +14116,0 +14117,0 +14118,0 +14119,0 +14120,0 +14121,0 +14122,0 +14123,0 +14124,0 +14125,0 +14126,0 +14127,0 +14128,0 +14129,0 +14130,0 +14131,0 +14132,0 +14133,0 +14134,0 +14135,0 +14136,0 +14137,0 +14138,0 +14139,0 +14140,0 +14141,0 +14142,0 +14143,0 +14144,0 +14145,0 +14146,0 +14147,0 +14148,0 +14149,0 +14150,0 +14151,0 +14152,0 +14153,0 +14154,0 +14155,0 +14156,0 +14157,0 +14158,0 +14159,0 +14160,0 +14161,0 +14162,0 +14163,0 +14164,0 +14165,0 +14166,0 +14167,0 +14168,0 +14169,0 +14170,0 +14171,0 +14172,0 +14173,0 +14174,0 +14175,0 +14176,0 +14177,0 +14178,0 +14179,0 +14180,0 +14181,0 +14182,0 +14183,0 +14184,0 +14185,0 +14186,0 +14187,0 +14188,0 +14189,0 +14190,0 +14191,0 +14192,0 +14193,0 +14194,0 +14195,0 +14196,0 +14197,0 +14198,0 +14199,0 +14200,0 +14201,0 +14202,0 +14203,0 +14204,0 +14205,0 +14206,0 +14207,0 +14208,0 +14209,0 +14210,0 +14211,0 +14212,0 +14213,0 +14214,0 +14215,0 +14216,0 +14217,0 +14218,0 +14219,0 +14220,0 +14221,0 +14222,0 +14223,0 +14224,0 +14225,0 +14226,0 +14227,0 +14228,0 +14229,0 +14230,0 +14231,0 +14232,0 +14233,0 +14234,0 +14235,0 +14236,0 +14237,0 +14238,0 +14239,0 +14240,0 +14241,0 +14242,0 +14243,0 +14244,0 +14245,0 +14246,0 +14247,0 +14248,0 +14249,0 +14250,0 +14251,0 +14252,0 +14253,0 +14254,0 +14255,0 +14256,0 +14257,0 +14258,0 +14259,0 +14260,0 +14261,0 +14262,0 +14263,0 +14264,0 +14265,0 +14266,0 +14267,0 +14268,0 +14269,0 +14270,0 +14271,0 +14272,0 +14273,0 +14274,0 +14275,0 +14276,0 +14277,0 +14278,0 +14279,0 +14280,0 +14281,0 +14282,0 +14283,0 +14284,0 +14285,0 +14286,0 +14287,0 +14288,0 +14289,0 +14290,0 +14291,0 +14292,0 +14293,0 +14294,0 +14295,0 +14296,0 +14297,0 +14298,0 +14299,0 +14300,0 +14301,0 +14302,0 +14303,0 +14304,0 +14305,0 +14306,0 +14307,0 +14308,0 +14309,0 +14310,0 +14311,0 +14312,0 +14313,0 +14314,0 +14315,0 +14316,0 +14317,0 +14318,0 +14319,0 +14320,0 +14321,0 +14322,0 +14323,0 +14324,0 +14325,0 +14326,0 +14327,0 +14328,0 +14329,0 +14330,0 +14331,0 +14332,0 +14333,0 +14334,0 +14335,0 +14336,0 +14337,0 +14338,0 +14339,0 +14340,0 +14341,0 +14342,0 +14343,0 +14344,0 +14345,0 +14346,0 +14347,0 +14348,0 +14349,0 +14350,0 +14351,0 +14352,0 +14353,0 +14354,0 +14355,0 +14356,0 +14357,0 +14358,0 +14359,0 +14360,0 +14361,0 +14362,0 +14363,0 +14364,0 +14365,0 +14366,0 +14367,0 +14368,0 +14369,0 +14370,0 +14371,0 +14372,0 +14373,0 +14374,0 +14375,0 +14376,0 +14377,0 +14378,0 +14379,0 +14380,0 +14381,0 +14382,0 +14383,0 +14384,0 +14385,0 +14386,0 +14387,0 +14388,0 +14389,0 +14390,0 +14391,0 +14392,0 +14393,0 +14394,0 +14395,0 +14396,0 +14397,0 +14398,0 +14399,0 +14400,0 +14401,0 +14402,0 +14403,0 +14404,0 +14405,0 +14406,0 +14407,0 +14408,0 +14409,0 +14410,0 +14411,0 +14412,0 +14413,0 +14414,0 +14415,0 +14416,0 +14417,0 +14418,0 +14419,0 +14420,0 +14421,0 +14422,0 +14423,0 +14424,0 +14425,0 +14426,0 +14427,0 +14428,0 +14429,0 +14430,0 +14431,0 +14432,0 +14433,0 +14434,0 +14435,0 +14436,0 +14437,0 +14438,0 +14439,0 +14440,0 +14441,0 +14442,0 +14443,0 +14444,0 +14445,0 +14446,0 +14447,0 +14448,0 +14449,0 +14450,0 +14451,0 +14452,0 +14453,0 +14454,0 +14455,0 +14456,0 +14457,0 +14458,0 +14459,0 +14460,0 +14461,0 +14462,0 +14463,0 +14464,0 +14465,0 +14466,0 +14467,0 +14468,0 +14469,0 +14470,0 +14471,0 +14472,0 +14473,0 +14474,0 +14475,0 +14476,0 +14477,0 +14478,0 +14479,0 +14480,0 +14481,0 +14482,0 +14483,0 +14484,0 +14485,0 +14486,0 +14487,0 +14488,0 +14489,0 +14490,0 +14491,0 +14492,0 +14493,0 +14494,0 +14495,0 +14496,0 +14497,0 +14498,0 +14499,0 +14500,0 +14501,0 +14502,0 +14503,0 +14504,0 +14505,0 +14506,0 +14507,0 +14508,0 +14509,0 +14510,0 +14511,0 +14512,0 +14513,0 +14514,0 +14515,0 +14516,0 +14517,0 +14518,0 +14519,0 +14520,0 +14521,0 +14522,0 +14523,0 +14524,0 +14525,0 +14526,0 +14527,0 +14528,0 +14529,0 +14530,0 +14531,0 +14532,0 +14533,0 +14534,0 +14535,0 +14536,0 +14537,0 +14538,0 +14539,0 +14540,0 +14541,0 +14542,0 +14543,0 +14544,0 +14545,0 +14546,0 +14547,0 +14548,0 +14549,0 +14550,0 +14551,0 +14552,0 +14553,0 +14554,0 +14555,0 +14556,0 +14557,0 +14558,0 +14559,0 +14560,0 +14561,0 +14562,0 +14563,0 +14564,0 +14565,0 +14566,0 +14567,0 +14568,0 +14569,0 +14570,0 +14571,0 +14572,0 +14573,0 +14574,0 +14575,0 +14576,0 +14577,0 +14578,0 +14579,0 +14580,0 +14581,0 +14582,0 +14583,0 +14584,0 +14585,0 +14586,0 +14587,0 +14588,0 +14589,0 +14590,0 +14591,0 +14592,0 +14593,0 +14594,0 +14595,0 +14596,0 +14597,0 +14598,0 +14599,0 +14600,0 +14601,0 +14602,0 +14603,0 +14604,0 +14605,0 +14606,0 +14607,0 +14608,0 +14609,0 +14610,0 +14611,0 +14612,0 +14613,0 +14614,0 +14615,0 +14616,0 +14617,0 +14618,0 +14619,0 +14620,0 +14621,0 +14622,0 +14623,0 +14624,0 +14625,0 +14626,0 +14627,0 +14628,0 +14629,0 +14630,0 +14631,0 +14632,0 +14633,0 +14634,0 +14635,0 +14636,0 +14637,0 +14638,0 +14639,0 +14640,0 +14641,0 +14642,0 +14643,0 +14644,0 +14645,0 +14646,0 +14647,0 +14648,0 +14649,0 +14650,0 +14651,0 +14652,0 +14653,0 +14654,0 +14655,0 +14656,0 +14657,0 +14658,0 +14659,0 +14660,0 +14661,0 +14662,0 +14663,0 +14664,0 +14665,0 +14666,0 +14667,0 +14668,0 +14669,0 +14670,0 +14671,0 +14672,0 +14673,0 +14674,0 +14675,0 +14676,0 +14677,0 +14678,0 +14679,0 +14680,0 +14681,0 +14682,0 +14683,0 +14684,0 +14685,0 +14686,0 +14687,0 +14688,0 +14689,0 +14690,0 +14691,0 +14692,0 +14693,0 +14694,0 +14695,0 +14696,0 +14697,0 +14698,0 +14699,0 +14700,0 +14701,0 +14702,0 +14703,0 +14704,0 +14705,0 +14706,0 +14707,0 +14708,0 +14709,0 +14710,0 +14711,0 +14712,0 +14713,0 +14714,0 +14715,0 +14716,0 +14717,0 +14718,0 +14719,0 +14720,0 +14721,0 +14722,0 +14723,0 +14724,0 +14725,0 +14726,0 +14727,0 +14728,0 +14729,0 +14730,0 +14731,0 +14732,0 +14733,0 +14734,0 +14735,0 +14736,0 +14737,0 +14738,0 +14739,0 +14740,0 +14741,0 +14742,0 +14743,0 +14744,0 +14745,0 +14746,0 +14747,0 +14748,0 +14749,0 +14750,0 +14751,0 +14752,0 +14753,0 +14754,0 +14755,0 +14756,0 +14757,0 +14758,0 +14759,0 +14760,0 +14761,0 +14762,0 +14763,0 +14764,0 +14765,0 +14766,0 +14767,0 +14768,0 +14769,0 +14770,0 +14771,0 +14772,0 +14773,0 +14774,0 +14775,0 +14776,0 +14777,0 +14778,0 +14779,0 +14780,0 +14781,0 +14782,0 +14783,0 +14784,0 +14785,0 +14786,0 +14787,0 +14788,0 +14789,0 +14790,0 +14791,0 +14792,0 +14793,0 +14794,0 +14795,0 +14796,0 +14797,0 +14798,0 +14799,0 +14800,0 +14801,0 +14802,0 +14803,0 +14804,0 +14805,0 +14806,0 +14807,0 +14808,0 +14809,0 +14810,0 +14811,0 +14812,0 +14813,0 +14814,0 +14815,0 +14816,0 +14817,0 +14818,0 +14819,0 +14820,0 +14821,0 +14822,0 +14823,0 +14824,0 +14825,0 +14826,0 +14827,0 +14828,0 +14829,0 +14830,0 +14831,0 +14832,0 +14833,0 +14834,0 +14835,0 +14836,0 +14837,0 +14838,0 +14839,0 +14840,0 +14841,0 +14842,0 +14843,0 +14844,0 +14845,0 +14846,0 +14847,0 +14848,0 +14849,0 +14850,0 +14851,0 +14852,0 +14853,0 +14854,0 +14855,0 +14856,0 +14857,0 +14858,0 +14859,0 +14860,0 +14861,0 +14862,0 +14863,0 +14864,0 +14865,0 +14866,0 +14867,0 +14868,0 +14869,0 +14870,0 +14871,0 +14872,0 +14873,0 +14874,0 +14875,0 +14876,0 +14877,0 +14878,0 +14879,0 +14880,0 +14881,0 +14882,0 +14883,0 +14884,0 +14885,0 +14886,0 +14887,0 +14888,0 +14889,0 +14890,0 +14891,0 +14892,0 +14893,0 +14894,0 +14895,0 +14896,0 +14897,0 +14898,0 +14899,0 +14900,0 +14901,0 +14902,0 +14903,0 +14904,0 +14905,0 +14906,0 +14907,0 +14908,0 +14909,0 +14910,0 +14911,0 +14912,0 +14913,0 +14914,0 +14915,0 +14916,0 +14917,0 +14918,0 +14919,0 +14920,0 +14921,0 +14922,0 +14923,0 +14924,0 +14925,0 +14926,0 +14927,0 +14928,0 +14929,0 +14930,0 +14931,0 +14932,0 +14933,0 +14934,0 +14935,0 +14936,0 +14937,0 +14938,0 +14939,0 +14940,0 +14941,0 +14942,0 +14943,0 +14944,0 +14945,0 +14946,0 +14947,0 +14948,0 +14949,0 +14950,0 +14951,0 +14952,0 +14953,0 +14954,0 +14955,0 +14956,0 +14957,0 +14958,0 +14959,0 +14960,0 +14961,0 +14962,0 +14963,0 +14964,0 +14965,0 +14966,0 +14967,0 +14968,0 +14969,0 +14970,0 +14971,0 +14972,0 +14973,0 +14974,0 +14975,0 +14976,0 +14977,0 +14978,0 +14979,0 +14980,0 +14981,0 +14982,0 +14983,0 +14984,0 +14985,0 +14986,0 +14987,0 +14988,0 +14989,0 +14990,0 +14991,0 +14992,0 +14993,0 +14994,0 +14995,0 +14996,0 +14997,0 +14998,0 +14999,0 +15000,0 +15001,0 +15002,0 +15003,0 +15004,0 +15005,0 +15006,0 +15007,0 +15008,0 +15009,0 +15010,0 +15011,0 +15012,0 +15013,0 +15014,0 +15015,0 +15016,0 +15017,0 +15018,0 +15019,0 +15020,0 +15021,0 +15022,0 +15023,0 +15024,0 +15025,0 +15026,0 +15027,0 +15028,0 +15029,0 +15030,0 +15031,0 +15032,0 +15033,0 +15034,0 +15035,0 +15036,0 +15037,0 +15038,0 +15039,0 +15040,0 +15041,0 +15042,0 +15043,0 +15044,0 +15045,0 +15046,0 +15047,0 +15048,0 +15049,0 +15050,0 +15051,0 +15052,0 +15053,0 +15054,0 +15055,0 +15056,0 +15057,0 +15058,0 +15059,0 +15060,0 +15061,0 +15062,0 +15063,0 +15064,0 +15065,0 +15066,0 +15067,0 +15068,0 +15069,0 +15070,0 +15071,0 +15072,0 +15073,0 +15074,0 +15075,0 +15076,0 +15077,0 +15078,0 +15079,0 +15080,0 +15081,0 +15082,0 +15083,0 +15084,0 +15085,0 +15086,0 +15087,0 +15088,0 +15089,0 +15090,0 +15091,0 +15092,0 +15093,0 +15094,0 +15095,0 +15096,0 +15097,0 +15098,0 +15099,0 +15100,0 +15101,0 +15102,0 +15103,0 +15104,0 +15105,0 +15106,0 +15107,0 +15108,0 +15109,0 +15110,0 +15111,0 +15112,0 +15113,0 +15114,0 +15115,0 +15116,0 +15117,0 +15118,0 +15119,0 +15120,0 +15121,0 +15122,0 +15123,0 +15124,0 +15125,0 +15126,0 +15127,0 +15128,0 +15129,0 +15130,0 +15131,0 +15132,0 +15133,0 +15134,0 +15135,0 +15136,0 +15137,0 +15138,0 +15139,0 +15140,0 +15141,0 +15142,0 +15143,0 +15144,0 +15145,0 +15146,0 +15147,0 +15148,0 +15149,0 +15150,0 +15151,0 +15152,0 +15153,0 +15154,0 +15155,0 +15156,0 +15157,0 +15158,0 +15159,0 +15160,0 +15161,0 +15162,0 +15163,0 +15164,0 +15165,0 +15166,0 +15167,0 +15168,0 +15169,0 +15170,0 +15171,0 +15172,0 +15173,0 +15174,0 +15175,0 +15176,0 +15177,0 +15178,0 +15179,0 +15180,0 +15181,0 +15182,0 +15183,0 +15184,0 +15185,0 +15186,0 +15187,0 +15188,0 +15189,0 +15190,0 +15191,0 +15192,0 +15193,0 +15194,0 +15195,0 +15196,0 +15197,0 +15198,0 +15199,0 +15200,0 +15201,0 +15202,0 +15203,0 +15204,0 +15205,0 +15206,0 +15207,0 +15208,0 +15209,0 +15210,0 +15211,0 +15212,0 +15213,0 +15214,0 +15215,0 +15216,0 +15217,0 +15218,0 +15219,0 +15220,0 +15221,0 +15222,0 +15223,0 +15224,0 +15225,0 +15226,0 +15227,0 +15228,0 +15229,0 +15230,0 +15231,0 +15232,0 +15233,0 +15234,0 +15235,0 +15236,0 +15237,0 +15238,0 +15239,0 +15240,0 +15241,0 +15242,0 +15243,0 +15244,0 +15245,0 +15246,0 +15247,0 +15248,0 +15249,0 +15250,0 +15251,0 +15252,0 +15253,0 +15254,0 +15255,0 +15256,0 +15257,0 +15258,0 +15259,0 +15260,0 +15261,0 +15262,0 +15263,0 +15264,0 +15265,0 +15266,0 +15267,0 +15268,0 +15269,0 +15270,0 +15271,0 +15272,0 +15273,0 +15274,0 +15275,0 +15276,0 +15277,0 +15278,0 +15279,0 +15280,0 +15281,0 +15282,0 +15283,0 +15284,0 +15285,0 +15286,0 +15287,0 +15288,0 +15289,0 +15290,0 +15291,0 +15292,0 +15293,0 +15294,0 +15295,0 +15296,0 +15297,0 +15298,0 +15299,0 +15300,0 +15301,0 +15302,0 +15303,0 +15304,0 +15305,0 +15306,0 +15307,0 +15308,0 +15309,0 +15310,0 +15311,0 +15312,0 +15313,0 +15314,0 +15315,0 +15316,0 +15317,0 +15318,0 +15319,0 +15320,0 +15321,0 +15322,0 +15323,0 +15324,0 +15325,0 +15326,0 +15327,0 +15328,0 +15329,0 +15330,0 +15331,0 +15332,0 +15333,0 +15334,0 +15335,0 +15336,0 +15337,0 +15338,0 +15339,0 +15340,0 +15341,0 +15342,0 +15343,0 +15344,0 +15345,0 +15346,0 +15347,0 +15348,0 +15349,0 +15350,0 +15351,0 +15352,0 +15353,0 +15354,0 +15355,0 +15356,0 +15357,0 +15358,0 +15359,0 +15360,0 +15361,0 +15362,0 +15363,0 +15364,0 +15365,0 +15366,0 +15367,0 +15368,0 +15369,0 +15370,0 +15371,0 +15372,0 +15373,0 +15374,0 +15375,0 +15376,0 +15377,0 +15378,0 +15379,0 +15380,0 +15381,0 +15382,0 +15383,0 +15384,0 +15385,0 +15386,0 +15387,0 +15388,0 +15389,0 +15390,0 +15391,0 +15392,0 +15393,0 +15394,0 +15395,0 +15396,0 +15397,0 +15398,0 +15399,0 +15400,0 +15401,0 +15402,0 +15403,0 +15404,0 +15405,0 +15406,0 +15407,0 +15408,0 +15409,0 +15410,0 +15411,0 +15412,0 +15413,0 +15414,0 +15415,0 +15416,0 +15417,0 +15418,0 +15419,0 +15420,0 +15421,0 +15422,0 +15423,0 +15424,0 +15425,0 +15426,0 +15427,0 +15428,0 +15429,0 +15430,0 +15431,0 +15432,0 +15433,0 +15434,0 +15435,0 +15436,0 +15437,0 +15438,0 +15439,0 +15440,0 +15441,0 +15442,0 +15443,0 +15444,0 +15445,0 +15446,0 +15447,0 +15448,0 +15449,0 +15450,0 +15451,0 +15452,0 +15453,0 +15454,0 +15455,0 +15456,0 +15457,0 +15458,0 +15459,0 +15460,0 +15461,0 +15462,0 +15463,0 +15464,0 +15465,0 +15466,0 +15467,0 +15468,0 +15469,0 +15470,0 +15471,0 +15472,0 +15473,0 +15474,0 +15475,0 +15476,0 +15477,0 +15478,0 +15479,0 +15480,0 +15481,0 +15482,0 +15483,0 +15484,0 +15485,0 +15486,0 +15487,0 +15488,0 +15489,0 +15490,0 +15491,0 +15492,0 +15493,0 +15494,0 +15495,0 +15496,0 +15497,0 +15498,0 +15499,0 +15500,0 +15501,0 +15502,0 +15503,0 +15504,0 +15505,0 +15506,0 +15507,0 +15508,0 +15509,0 +15510,0 +15511,0 +15512,0 +15513,0 +15514,0 +15515,0 +15516,0 +15517,0 +15518,0 +15519,0 +15520,0 +15521,0 +15522,0 +15523,0 +15524,0 +15525,0 +15526,0 +15527,0 +15528,0 +15529,0 +15530,0 +15531,0 +15532,0 +15533,0 +15534,0 +15535,0 +15536,0 +15537,0 +15538,0 +15539,0 +15540,0 +15541,0 +15542,0 +15543,0 +15544,0 +15545,0 +15546,0 +15547,0 +15548,0 +15549,0 +15550,0 +15551,0 +15552,0 +15553,0 +15554,0 +15555,0 +15556,0 +15557,0 +15558,0 +15559,0 +15560,0 +15561,0 +15562,0 +15563,0 +15564,0 +15565,0 +15566,0 +15567,0 +15568,0 +15569,0 +15570,0 +15571,0 +15572,0 +15573,0 +15574,0 +15575,0 +15576,0 +15577,0 +15578,0 +15579,0 +15580,0 +15581,0 +15582,0 +15583,0 +15584,0 +15585,0 +15586,0 +15587,0 +15588,0 +15589,0 +15590,0 +15591,0 +15592,0 +15593,0 +15594,0 +15595,0 +15596,0 +15597,0 +15598,0 +15599,0 +15600,0 +15601,0 +15602,0 +15603,0 +15604,0 +15605,0 +15606,0 +15607,0 +15608,0 +15609,0 +15610,0 +15611,0 +15612,0 +15613,0 +15614,0 +15615,0 +15616,0 +15617,0 +15618,0 +15619,0 +15620,0 +15621,0 +15622,0 +15623,0 +15624,0 +15625,0 +15626,0 +15627,0 +15628,0 +15629,0 +15630,0 +15631,0 +15632,0 +15633,0 +15634,0 +15635,0 +15636,0 +15637,0 +15638,0 +15639,0 +15640,0 +15641,0 +15642,0 +15643,0 +15644,0 +15645,0 +15646,0 +15647,0 +15648,0 +15649,0 +15650,0 +15651,0 +15652,0 +15653,0 +15654,0 +15655,0 +15656,0 +15657,0 +15658,0 +15659,0 +15660,0 +15661,0 +15662,0 +15663,0 +15664,0 +15665,0 +15666,0 +15667,0 +15668,0 +15669,0 +15670,0 +15671,0 +15672,0 +15673,0 +15674,0 +15675,0 +15676,0 +15677,0 +15678,0 +15679,0 +15680,0 +15681,0 +15682,0 +15683,0 +15684,0 +15685,0 +15686,0 +15687,0 +15688,0 +15689,0 +15690,0 +15691,0 +15692,0 +15693,0 +15694,0 +15695,0 +15696,0 +15697,0 +15698,0 +15699,0 +15700,0 +15701,0 +15702,0 +15703,0 +15704,0 +15705,0 +15706,0 +15707,0 +15708,0 +15709,0 +15710,0 +15711,0 +15712,0 +15713,0 +15714,0 +15715,0 +15716,0 +15717,0 +15718,0 +15719,0 +15720,0 +15721,0 +15722,0 +15723,0 +15724,0 +15725,0 +15726,0 +15727,0 +15728,0 +15729,0 +15730,0 +15731,0 +15732,0 +15733,0 +15734,0 +15735,0 +15736,0 +15737,0 +15738,0 +15739,0 +15740,0 +15741,0 +15742,0 +15743,0 +15744,0 +15745,0 +15746,0 +15747,0 +15748,0 +15749,0 +15750,0 +15751,0 +15752,0 +15753,0 +15754,0 +15755,0 +15756,0 +15757,0 +15758,0 +15759,0 +15760,0 +15761,0 +15762,0 +15763,0 +15764,0 +15765,0 +15766,0 +15767,0 +15768,0 +15769,0 +15770,0 +15771,0 +15772,0 +15773,0 +15774,0 +15775,0 +15776,0 +15777,0 +15778,0 +15779,0 +15780,0 +15781,0 +15782,0 +15783,0 +15784,0 +15785,0 +15786,0 +15787,0 +15788,0 +15789,0 +15790,0 +15791,0 +15792,0 +15793,0 +15794,0 +15795,0 +15796,0 +15797,0 +15798,0 +15799,0 +15800,0 +15801,0 +15802,0 +15803,0 +15804,0 +15805,0 +15806,0 +15807,0 +15808,0 +15809,0 +15810,0 +15811,0 +15812,0 +15813,0 +15814,0 +15815,0 +15816,0 +15817,0 +15818,0 +15819,0 +15820,0 +15821,0 +15822,0 +15823,0 +15824,0 +15825,0 +15826,0 +15827,0 +15828,0 +15829,0 +15830,0 +15831,0 +15832,0 +15833,0 +15834,0 +15835,0 +15836,0 +15837,0 +15838,0 +15839,0 +15840,0 +15841,0 +15842,0 +15843,0 +15844,0 +15845,0 +15846,0 +15847,0 +15848,0 +15849,0 +15850,0 +15851,0 +15852,0 +15853,0 +15854,0 +15855,0 +15856,0 +15857,0 +15858,0 +15859,0 +15860,0 +15861,0 +15862,0 +15863,0 +15864,0 +15865,0 +15866,0 +15867,0 +15868,0 +15869,0 +15870,0 +15871,0 +15872,0 +15873,0 +15874,0 +15875,0 +15876,0 +15877,0 +15878,0 +15879,0 +15880,0 +15881,0 +15882,0 +15883,0 +15884,0 +15885,0 +15886,0 +15887,0 +15888,0 +15889,0 +15890,0 +15891,0 +15892,0 +15893,0 +15894,0 +15895,0 +15896,0 +15897,0 +15898,0 +15899,0 +15900,0 +15901,0 +15902,0 +15903,0 +15904,0 +15905,0 +15906,0 +15907,0 +15908,0 +15909,0 +15910,0 +15911,0 +15912,0 +15913,0 +15914,0 +15915,0 +15916,0 +15917,0 +15918,0 +15919,0 +15920,0 +15921,0 +15922,0 +15923,0 +15924,0 +15925,0 +15926,0 +15927,0 +15928,0 +15929,0 +15930,0 +15931,0 +15932,0 +15933,0 +15934,0 +15935,0 +15936,0 +15937,0 +15938,0 +15939,0 +15940,0 +15941,0 +15942,0 +15943,0 +15944,0 +15945,0 +15946,0 +15947,0 +15948,0 +15949,0 +15950,0 +15951,0 +15952,0 +15953,0 +15954,0 +15955,0 +15956,0 +15957,0 +15958,0 +15959,0 +15960,0 +15961,0 +15962,0 +15963,0 +15964,0 +15965,0 +15966,0 +15967,0 +15968,0 +15969,0 +15970,0 +15971,0 +15972,0 +15973,0 +15974,0 +15975,0 +15976,0 +15977,0 +15978,0 +15979,0 +15980,0 +15981,0 +15982,0 +15983,0 +15984,0 +15985,0 +15986,0 +15987,0 +15988,0 +15989,0 +15990,0 +15991,0 +15992,0 +15993,0 +15994,0 +15995,0 +15996,0 +15997,0 +15998,0 +15999,0 +16000,0 +16001,0 +16002,0 +16003,0 +16004,0 +16005,0 +16006,0 +16007,0 +16008,0 +16009,0 +16010,0 +16011,0 +16012,0 +16013,0 +16014,0 +16015,0 +16016,0 +16017,0 +16018,0 +16019,0 +16020,0 +16021,0 +16022,0 +16023,0 +16024,0 +16025,0 +16026,0 +16027,0 +16028,0 +16029,0 +16030,0 +16031,0 +16032,0 +16033,0 +16034,0 +16035,0 +16036,0 +16037,0 +16038,0 +16039,0 +16040,0 +16041,0 +16042,0 +16043,0 +16044,0 +16045,0 +16046,0 +16047,0 +16048,0 +16049,0 +16050,0 +16051,0 +16052,0 +16053,0 +16054,0 +16055,0 +16056,0 +16057,0 +16058,0 +16059,0 +16060,0 +16061,0 +16062,0 +16063,0 +16064,0 +16065,0 +16066,0 +16067,0 +16068,0 +16069,0 +16070,0 +16071,0 +16072,0 +16073,0 +16074,0 +16075,0 +16076,0 +16077,0 +16078,0 +16079,0 +16080,0 +16081,0 +16082,0 +16083,0 +16084,0 +16085,0 +16086,0 +16087,0 +16088,0 +16089,0 +16090,0 +16091,0 +16092,0 +16093,0 +16094,0 +16095,0 +16096,0 +16097,0 +16098,0 +16099,0 +16100,0 +16101,0 +16102,0 +16103,0 +16104,0 +16105,0 +16106,0 +16107,0 +16108,0 +16109,0 +16110,0 +16111,0 +16112,0 +16113,0 +16114,0 +16115,0 +16116,0 +16117,0 +16118,0 +16119,0 +16120,0 +16121,0 +16122,0 +16123,0 +16124,0 +16125,0 +16126,0 +16127,0 +16128,0 +16129,0 +16130,0 +16131,0 +16132,0 +16133,0 +16134,0 +16135,0 +16136,0 +16137,0 +16138,0 +16139,0 +16140,0 +16141,0 +16142,0 +16143,0 +16144,0 +16145,0 +16146,0 +16147,0 +16148,0 +16149,0 +16150,0 +16151,0 +16152,0 +16153,0 +16154,0 +16155,0 +16156,0 +16157,0 +16158,0 +16159,0 +16160,0 +16161,0 +16162,0 +16163,0 +16164,0 +16165,0 +16166,0 +16167,0 +16168,0 +16169,0 +16170,0 +16171,0 +16172,0 +16173,0 +16174,0 +16175,0 +16176,0 +16177,0 +16178,0 +16179,0 +16180,0 +16181,0 +16182,0 +16183,0 +16184,0 +16185,0 +16186,0 +16187,0 +16188,0 +16189,0 +16190,0 +16191,0 +16192,0 +16193,0 +16194,0 +16195,0 +16196,0 +16197,0 +16198,0 +16199,0 +16200,0 +16201,0 +16202,0 +16203,0 +16204,0 +16205,0 +16206,0 +16207,0 +16208,0 +16209,0 +16210,0 +16211,0 +16212,0 +16213,0 +16214,0 +16215,0 +16216,0 +16217,0 +16218,0 +16219,0 +16220,0 +16221,0 +16222,0 +16223,0 +16224,0 +16225,0 +16226,0 +16227,0 +16228,0 +16229,0 +16230,0 +16231,0 +16232,0 +16233,0 +16234,0 +16235,0 +16236,0 +16237,0 +16238,0 +16239,0 +16240,0 +16241,0 +16242,0 +16243,0 +16244,0 +16245,0 +16246,0 +16247,0 +16248,0 +16249,0 +16250,0 +16251,0 +16252,0 +16253,0 +16254,0 +16255,0 +16256,0 +16257,0 +16258,0 +16259,0 +16260,0 +16261,0 +16262,0 +16263,0 +16264,0 +16265,0 +16266,0 +16267,0 +16268,0 +16269,0 +16270,0 +16271,0 +16272,0 +16273,0 +16274,0 +16275,0 +16276,0 +16277,0 +16278,0 +16279,0 +16280,0 +16281,0 +16282,0 +16283,0 +16284,0 +16285,0 +16286,0 +16287,0 +16288,0 +16289,0 +16290,0 +16291,0 +16292,0 +16293,0 +16294,0 +16295,0 +16296,0 +16297,0 +16298,0 +16299,0 +16300,0 +16301,0 +16302,0 +16303,0 +16304,0 +16305,0 +16306,0 +16307,0 +16308,0 +16309,0 +16310,0 +16311,0 +16312,0 +16313,0 +16314,0 +16315,0 +16316,0 +16317,0 +16318,0 +16319,0 +16320,0 +16321,0 +16322,0 +16323,0 +16324,0 +16325,0 +16326,0 +16327,0 +16328,0 +16329,0 +16330,0 +16331,0 +16332,0 +16333,0 +16334,0 +16335,0 +16336,0 +16337,0 +16338,0 +16339,0 +16340,0 +16341,0 +16342,0 +16343,0 +16344,0 +16345,0 +16346,0 +16347,0 +16348,0 +16349,0 +16350,0 +16351,0 +16352,0 +16353,0 +16354,0 +16355,0 +16356,0 +16357,0 +16358,0 +16359,0 +16360,0 +16361,0 +16362,0 +16363,0 +16364,0 +16365,0 +16366,0 +16367,0 +16368,0 +16369,0 +16370,0 +16371,0 +16372,0 +16373,0 +16374,0 +16375,0 +16376,0 +16377,0 +16378,0 +16379,0 +16380,0 +16381,0 +16382,0 +16383,0 +16384,0 +16385,0 +16386,0 +16387,0 +16388,0 +16389,0 +16390,0 +16391,0 +16392,0 +16393,0 +16394,0 +16395,0 +16396,0 +16397,0 +16398,0 +16399,0 +16400,0 +16401,0 +16402,0 +16403,0 +16404,0 +16405,0 +16406,0 +16407,0 +16408,0 +16409,0 +16410,0 +16411,0 +16412,0 +16413,0 +16414,0 +16415,0 +16416,0 +16417,0 +16418,0 +16419,0 +16420,0 +16421,0 +16422,0 +16423,0 +16424,0 +16425,0 +16426,0 +16427,0 +16428,0 +16429,0 +16430,0 +16431,0 +16432,0 +16433,0 +16434,0 +16435,0 +16436,0 +16437,0 +16438,0 +16439,0 +16440,0 +16441,0 +16442,0 +16443,0 +16444,0 +16445,0 +16446,0 +16447,0 +16448,0 +16449,0 +16450,0 +16451,0 +16452,0 +16453,0 +16454,0 +16455,0 +16456,0 +16457,0 +16458,0 +16459,0 +16460,0 +16461,0 +16462,0 +16463,0 +16464,0 +16465,0 +16466,0 +16467,0 +16468,0 +16469,0 +16470,0 +16471,0 +16472,0 +16473,0 +16474,0 +16475,0 +16476,0 +16477,0 +16478,0 +16479,0 +16480,0 +16481,0 +16482,0 +16483,0 +16484,0 +16485,0 +16486,0 +16487,0 +16488,0 +16489,0 +16490,0 +16491,0 +16492,0 +16493,0 +16494,0 +16495,0 +16496,0 +16497,0 +16498,0 +16499,0 +16500,0 +16501,0 +16502,0 +16503,0 +16504,0 +16505,0 +16506,0 +16507,0 +16508,0 +16509,0 +16510,0 +16511,0 +16512,0 +16513,0 +16514,0 +16515,0 +16516,0 +16517,0 +16518,0 +16519,0 +16520,0 +16521,0 +16522,0 +16523,0 +16524,0 +16525,0 +16526,0 +16527,0 +16528,0 +16529,0 +16530,0 +16531,0 +16532,0 +16533,0 +16534,0 +16535,0 +16536,0 +16537,0 +16538,0 +16539,0 +16540,0 +16541,0 +16542,0 +16543,0 +16544,0 +16545,0 +16546,0 +16547,0 +16548,0 +16549,0 +16550,0 +16551,0 +16552,0 +16553,0 +16554,0 +16555,0 +16556,0 +16557,0 +16558,0 +16559,0 +16560,0 +16561,0 +16562,0 +16563,0 +16564,0 +16565,0 +16566,0 +16567,0 +16568,0 +16569,0 +16570,0 +16571,0 +16572,0 +16573,0 +16574,0 +16575,0 +16576,0 +16577,0 +16578,0 +16579,0 +16580,0 +16581,0 +16582,0 +16583,0 +16584,0 +16585,0 +16586,0 +16587,0 +16588,0 +16589,0 +16590,0 +16591,0 +16592,0 +16593,0 +16594,0 +16595,0 +16596,0 +16597,0 +16598,0 +16599,0 +16600,0 +16601,0 +16602,0 +16603,0 +16604,0 +16605,0 +16606,0 +16607,0 +16608,0 +16609,0 +16610,0 +16611,0 +16612,0 +16613,0 +16614,0 +16615,0 +16616,0 +16617,0 +16618,0 +16619,0 +16620,0 +16621,0 +16622,0 +16623,0 +16624,0 +16625,0 +16626,0 +16627,0 +16628,0 +16629,0 +16630,0 +16631,0 +16632,0 +16633,0 +16634,0 +16635,0 +16636,0 +16637,0 +16638,0 +16639,0 +16640,0 +16641,0 +16642,0 +16643,0 +16644,0 +16645,0 +16646,0 +16647,0 +16648,0 +16649,0 +16650,0 +16651,0 +16652,0 +16653,0 +16654,0 +16655,0 +16656,0 +16657,0 +16658,0 +16659,0 +16660,0 +16661,0 +16662,0 +16663,0 +16664,0 +16665,0 +16666,0 +16667,0 +16668,0 +16669,0 +16670,0 +16671,0 +16672,0 +16673,0 +16674,0 +16675,0 +16676,0 +16677,0 +16678,0 +16679,0 +16680,0 +16681,0 +16682,0 +16683,0 +16684,0 +16685,0 +16686,0 +16687,0 +16688,0 +16689,0 +16690,0 +16691,0 +16692,0 +16693,0 +16694,0 +16695,0 +16696,0 +16697,0 +16698,0 +16699,0 +16700,0 +16701,0 +16702,0 +16703,0 +16704,0 +16705,0 +16706,0 +16707,0 +16708,0 +16709,0 +16710,0 +16711,0 +16712,0 +16713,0 +16714,0 +16715,0 +16716,0 +16717,0 +16718,0 +16719,0 +16720,0 +16721,0 +16722,0 +16723,0 +16724,0 +16725,0 +16726,0 +16727,0 +16728,0 +16729,0 +16730,0 +16731,0 +16732,0 +16733,0 +16734,0 +16735,0 +16736,0 +16737,0 +16738,0 +16739,0 +16740,0 +16741,0 +16742,0 +16743,0 +16744,0 +16745,0 +16746,0 +16747,0 +16748,0 +16749,0 +16750,0 +16751,0 +16752,0 +16753,0 +16754,0 +16755,0 +16756,0 +16757,0 +16758,0 +16759,0 +16760,0 +16761,0 +16762,0 +16763,0 +16764,0 +16765,0 +16766,0 +16767,0 +16768,0 +16769,0 +16770,0 +16771,0 +16772,0 +16773,0 +16774,0 +16775,0 +16776,0 +16777,0 +16778,0 +16779,0 +16780,0 +16781,0 +16782,0 +16783,0 +16784,0 +16785,0 +16786,0 +16787,0 +16788,0 +16789,0 +16790,0 +16791,0 +16792,0 +16793,0 +16794,0 +16795,0 +16796,0 +16797,0 +16798,0 +16799,0 +16800,0 +16801,0 +16802,0 +16803,0 +16804,0 +16805,0 +16806,0 +16807,0 +16808,0 +16809,0 +16810,0 +16811,0 +16812,0 +16813,0 +16814,0 +16815,0 +16816,0 +16817,0 +16818,0 +16819,0 +16820,0 +16821,0 +16822,0 +16823,0 +16824,0 +16825,0 +16826,0 +16827,0 +16828,0 +16829,0 +16830,0 +16831,0 +16832,0 +16833,0 +16834,0 +16835,0 +16836,0 +16837,0 +16838,0 +16839,0 +16840,0 +16841,0 +16842,0 +16843,0 +16844,0 +16845,0 +16846,0 +16847,0 +16848,0 +16849,0 +16850,0 +16851,0 +16852,0 +16853,0 +16854,0 +16855,0 +16856,0 +16857,0 +16858,0 +16859,0 +16860,0 +16861,0 +16862,0 +16863,0 +16864,0 +16865,0 +16866,0 +16867,0 +16868,0 +16869,0 +16870,0 +16871,0 +16872,0 +16873,0 +16874,0 +16875,0 +16876,0 +16877,0 +16878,0 +16879,0 +16880,0 +16881,0 +16882,0 +16883,0 +16884,0 +16885,0 +16886,0 +16887,0 +16888,0 +16889,0 +16890,0 +16891,0 +16892,0 +16893,0 +16894,0 +16895,0 +16896,0 +16897,0 +16898,0 +16899,0 +16900,0 +16901,0 +16902,0 +16903,0 +16904,0 +16905,0 +16906,0 +16907,0 +16908,0 +16909,0 +16910,0 +16911,0 +16912,0 +16913,0 +16914,0 +16915,0 +16916,0 +16917,0 +16918,0 +16919,0 +16920,0 +16921,0 +16922,0 +16923,0 +16924,0 +16925,0 +16926,0 +16927,0 +16928,0 +16929,0 +16930,0 +16931,0 +16932,0 +16933,0 +16934,0 +16935,0 +16936,0 +16937,0 +16938,0 +16939,0 +16940,0 +16941,0 +16942,0 +16943,0 +16944,0 +16945,0 +16946,0 +16947,0 +16948,0 +16949,0 +16950,0 +16951,0 +16952,0 +16953,0 +16954,0 +16955,0 +16956,0 +16957,0 +16958,0 +16959,0 +16960,0 +16961,0 +16962,0 +16963,0 +16964,0 +16965,0 +16966,0 +16967,0 +16968,0 +16969,0 +16970,0 +16971,0 +16972,0 +16973,0 +16974,0 +16975,0 +16976,0 +16977,0 +16978,0 +16979,0 +16980,0 +16981,0 +16982,0 +16983,0 +16984,0 +16985,0 +16986,0 +16987,0 +16988,0 +16989,0 +16990,0 +16991,0 +16992,0 +16993,0 +16994,0 +16995,0 +16996,0 +16997,0 +16998,0 +16999,0 +17000,0 +17001,0 +17002,0 +17003,0 +17004,0 +17005,0 +17006,0 +17007,0 +17008,0 +17009,0 +17010,0 +17011,0 +17012,0 +17013,0 +17014,0 +17015,0 +17016,0 +17017,0 +17018,0 +17019,0 +17020,0 +17021,0 +17022,0 +17023,0 +17024,0 +17025,0 +17026,0 +17027,0 +17028,0 +17029,0 +17030,0 +17031,0 +17032,0 +17033,0 +17034,0 +17035,0 +17036,0 +17037,0 +17038,0 +17039,0 +17040,0 +17041,0 +17042,0 +17043,0 +17044,0 +17045,0 +17046,0 +17047,0 +17048,0 +17049,0 +17050,0 +17051,0 +17052,0 +17053,0 +17054,0 +17055,0 +17056,0 +17057,0 +17058,0 +17059,0 +17060,0 +17061,0 +17062,0 +17063,0 +17064,0 +17065,0 +17066,0 +17067,0 +17068,0 +17069,0 +17070,0 +17071,0 +17072,0 +17073,0 +17074,0 +17075,0 +17076,0 +17077,0 +17078,0 +17079,0 +17080,0 +17081,0 +17082,0 +17083,0 +17084,0 +17085,0 +17086,0 +17087,0 +17088,0 +17089,0 +17090,0 +17091,0 +17092,0 +17093,0 +17094,0 +17095,0 +17096,0 +17097,0 +17098,0 +17099,0 +17100,0 +17101,0 +17102,0 +17103,0 +17104,0 +17105,0 +17106,0 +17107,0 +17108,0 +17109,0 +17110,0 +17111,0 +17112,0 +17113,0 +17114,0 +17115,0 +17116,0 +17117,0 +17118,0 +17119,0 +17120,0 +17121,0 +17122,0 +17123,0 +17124,0 +17125,0 +17126,0 +17127,0 +17128,0 +17129,0 +17130,0 +17131,0 +17132,0 +17133,0 +17134,0 +17135,0 +17136,0 +17137,0 +17138,0 +17139,0 +17140,0 +17141,0 +17142,0 +17143,0 +17144,0 +17145,0 +17146,0 +17147,0 +17148,0 +17149,0 +17150,0 +17151,0 +17152,0 +17153,0 +17154,0 +17155,0 +17156,0 +17157,0 +17158,0 +17159,0 +17160,0 +17161,0 +17162,0 +17163,0 +17164,0 +17165,0 +17166,0 +17167,0 +17168,0 +17169,0 +17170,0 +17171,0 +17172,0 +17173,0 +17174,0 +17175,0 +17176,0 +17177,0 +17178,0 +17179,0 +17180,0 +17181,0 +17182,0 +17183,0 +17184,0 +17185,0 +17186,0 +17187,0 +17188,0 +17189,0 +17190,0 +17191,0 +17192,0 +17193,0 +17194,0 +17195,0 +17196,0 +17197,0 +17198,0 +17199,0 +17200,0 +17201,0 +17202,0 +17203,0 +17204,0 +17205,0 +17206,0 +17207,0 +17208,0 +17209,0 +17210,0 +17211,0 +17212,0 +17213,0 +17214,0 +17215,0 +17216,0 +17217,0 +17218,0 +17219,0 +17220,0 +17221,0 +17222,0 +17223,0 +17224,0 +17225,0 +17226,0 +17227,0 +17228,0 +17229,0 +17230,0 +17231,0 +17232,0 +17233,0 +17234,0 +17235,0 +17236,0 +17237,0 +17238,0 +17239,0 +17240,0 +17241,0 +17242,0 +17243,0 +17244,0 +17245,0 +17246,0 +17247,0 +17248,0 +17249,0 +17250,0 +17251,0 +17252,0 +17253,0 +17254,0 +17255,0 +17256,0 +17257,0 +17258,0 +17259,0 +17260,0 +17261,0 +17262,0 +17263,0 +17264,0 +17265,0 +17266,0 +17267,0 +17268,0 +17269,0 +17270,0 +17271,0 +17272,0 +17273,0 +17274,0 +17275,0 +17276,0 +17277,0 +17278,0 +17279,0 +17280,0 +17281,0 +17282,0 +17283,0 +17284,0 +17285,0 +17286,0 +17287,0 +17288,0 +17289,0 +17290,0 +17291,0 +17292,0 +17293,0 +17294,0 +17295,0 +17296,0 +17297,0 +17298,0 +17299,0 +17300,0 +17301,0 +17302,0 +17303,0 +17304,0 +17305,0 +17306,0 +17307,0 +17308,0 +17309,0 +17310,0 +17311,0 +17312,0 +17313,0 +17314,0 +17315,0 +17316,0 +17317,0 +17318,0 +17319,0 +17320,0 +17321,0 +17322,0 +17323,0 +17324,0 +17325,0 +17326,0 +17327,0 +17328,0 +17329,0 +17330,0 +17331,0 +17332,0 +17333,0 +17334,0 +17335,0 +17336,0 +17337,0 +17338,0 +17339,0 +17340,0 +17341,0 +17342,0 +17343,0 +17344,0 +17345,0 +17346,0 +17347,0 +17348,0 +17349,0 +17350,0 +17351,0 +17352,0 +17353,0 +17354,0 +17355,0 +17356,0 +17357,0 +17358,0 +17359,0 +17360,0 +17361,0 +17362,0 +17363,0 +17364,0 +17365,0 +17366,0 +17367,0 +17368,0 +17369,0 +17370,0 +17371,0 +17372,0 +17373,0 +17374,0 +17375,0 +17376,0 +17377,0 +17378,0 +17379,0 +17380,0 +17381,0 +17382,0 +17383,0 +17384,0 +17385,0 +17386,0 +17387,0 +17388,0 +17389,0 +17390,0 +17391,0 +17392,0 +17393,0 +17394,0 +17395,0 +17396,0 +17397,0 +17398,0 +17399,0 +17400,0 +17401,0 +17402,0 +17403,0 +17404,0 +17405,0 +17406,0 +17407,0 +17408,0 +17409,0 +17410,0 +17411,0 +17412,0 +17413,0 +17414,0 +17415,0 +17416,0 +17417,0 +17418,0 +17419,0 +17420,0 +17421,0 +17422,0 +17423,0 +17424,0 +17425,0 +17426,0 +17427,0 +17428,0 +17429,0 +17430,0 +17431,0 +17432,0 +17433,0 +17434,0 +17435,0 +17436,0 +17437,0 +17438,0 +17439,0 +17440,0 +17441,0 +17442,0 +17443,0 +17444,0 +17445,0 +17446,0 +17447,0 +17448,0 +17449,0 +17450,0 +17451,0 +17452,0 +17453,0 +17454,0 +17455,0 +17456,0 +17457,0 +17458,0 +17459,0 +17460,0 +17461,0 +17462,0 +17463,0 +17464,0 +17465,0 +17466,0 +17467,0 +17468,0 +17469,0 +17470,0 +17471,0 +17472,0 +17473,0 +17474,0 +17475,0 +17476,0 +17477,0 +17478,0 +17479,0 +17480,0 +17481,0 +17482,0 +17483,0 +17484,0 +17485,0 +17486,0 +17487,0 +17488,0 +17489,0 +17490,0 +17491,0 +17492,0 +17493,0 +17494,0 +17495,0 +17496,0 +17497,0 +17498,0 +17499,0 +17500,0 +17501,0 +17502,0 +17503,0 +17504,0 +17505,0 +17506,0 +17507,0 +17508,0 +17509,0 +17510,0 +17511,0 +17512,0 +17513,0 +17514,0 +17515,0 +17516,0 +17517,0 +17518,0 +17519,0 +17520,0 +17521,0 +17522,0 +17523,0 +17524,0 +17525,0 +17526,0 +17527,0 +17528,0 +17529,0 +17530,0 +17531,0 +17532,0 +17533,0 +17534,0 +17535,0 +17536,0 +17537,0 +17538,0 +17539,0 +17540,0 +17541,0 +17542,0 +17543,0 +17544,0 +17545,0 +17546,0 +17547,0 +17548,0 +17549,0 +17550,0 +17551,0 +17552,0 +17553,0 +17554,0 +17555,0 +17556,0 +17557,0 +17558,0 +17559,0 +17560,0 +17561,0 +17562,0 +17563,0 +17564,0 +17565,0 +17566,0 +17567,0 +17568,0 +17569,0 +17570,0 +17571,0 +17572,0 +17573,0 +17574,0 +17575,0 +17576,0 +17577,0 +17578,0 +17579,0 +17580,0 +17581,0 +17582,0 +17583,0 +17584,0 +17585,0 +17586,0 +17587,0 +17588,0 +17589,0 +17590,0 +17591,0 +17592,0 +17593,0 +17594,0 +17595,0 +17596,0 +17597,0 +17598,0 +17599,0 +17600,0 +17601,0 +17602,0 +17603,0 +17604,0 +17605,0 +17606,0 +17607,0 +17608,0 +17609,0 +17610,0 +17611,0 +17612,0 +17613,0 +17614,0 +17615,0 +17616,0 +17617,0 +17618,0 +17619,0 +17620,0 +17621,0 +17622,0 +17623,0 +17624,0 +17625,0 +17626,0 +17627,0 +17628,0 +17629,0 +17630,0 +17631,0 +17632,0 +17633,0 +17634,0 +17635,0 +17636,0 +17637,0 +17638,0 +17639,0 +17640,0 +17641,0 +17642,0 +17643,0 +17644,0 +17645,0 +17646,0 +17647,0 +17648,0 +17649,0 +17650,0 +17651,0 +17652,0 +17653,0 +17654,0 +17655,0 +17656,0 +17657,0 +17658,0 +17659,0 +17660,0 +17661,0 +17662,0 +17663,0 +17664,0 +17665,0 +17666,0 +17667,0 +17668,0 +17669,0 +17670,0 +17671,0 +17672,0 +17673,0 +17674,0 +17675,0 +17676,0 +17677,0 +17678,0 +17679,0 +17680,0 +17681,0 +17682,0 +17683,0 +17684,0 +17685,0 +17686,0 +17687,0 +17688,0 +17689,0 +17690,0 +17691,0 +17692,0 +17693,0 +17694,0 +17695,0 +17696,0 +17697,0 +17698,0 +17699,0 +17700,0 +17701,0 +17702,0 +17703,0 +17704,0 +17705,0 +17706,0 +17707,0 +17708,0 +17709,0 +17710,0 +17711,0 +17712,0 +17713,0 +17714,0 +17715,0 +17716,0 +17717,0 +17718,0 +17719,0 +17720,0 +17721,0 +17722,0 +17723,0 +17724,0 +17725,0 +17726,0 +17727,0 +17728,0 +17729,0 +17730,0 +17731,0 +17732,0 +17733,0 +17734,0 +17735,0 +17736,0 +17737,0 +17738,0 +17739,0 +17740,0 +17741,0 +17742,0 +17743,0 +17744,0 +17745,0 +17746,0 +17747,0 +17748,0 +17749,0 +17750,0 +17751,0 +17752,0 +17753,0 +17754,0 +17755,0 +17756,0 +17757,0 +17758,0 +17759,0 +17760,0 +17761,0 +17762,0 +17763,0 +17764,0 +17765,0 +17766,0 +17767,0 +17768,0 +17769,0 +17770,0 +17771,0 +17772,0 +17773,0 +17774,0 +17775,0 +17776,0 +17777,0 +17778,0 +17779,0 +17780,0 +17781,0 +17782,0 +17783,0 +17784,0 +17785,0 +17786,0 +17787,0 +17788,0 +17789,0 +17790,0 +17791,0 +17792,0 +17793,0 +17794,0 +17795,0 +17796,0 +17797,0 +17798,0 +17799,0 +17800,0 +17801,0 +17802,0 +17803,0 +17804,0 +17805,0 +17806,0 +17807,0 +17808,0 +17809,0 +17810,0 +17811,0 +17812,0 +17813,0 +17814,0 +17815,0 +17816,0 +17817,0 +17818,0 +17819,0 +17820,0 +17821,0 +17822,0 +17823,0 +17824,0 +17825,0 +17826,0 +17827,0 +17828,0 +17829,0 +17830,0 +17831,0 +17832,0 +17833,0 +17834,0 +17835,0 +17836,0 +17837,0 +17838,0 +17839,0 +17840,0 +17841,0 +17842,0 +17843,0 +17844,0 +17845,0 +17846,0 +17847,0 +17848,0 +17849,0 +17850,0 +17851,0 +17852,0 +17853,0 +17854,0 +17855,0 +17856,0 +17857,0 +17858,0 +17859,0 +17860,0 +17861,0 +17862,0 +17863,0 +17864,0 +17865,0 +17866,0 +17867,0 +17868,0 +17869,0 +17870,0 +17871,0 +17872,0 +17873,0 +17874,0 +17875,0 +17876,0 +17877,0 +17878,0 +17879,0 +17880,0 +17881,0 +17882,0 +17883,0 +17884,0 +17885,0 +17886,0 +17887,0 +17888,0 +17889,0 +17890,0 +17891,0 +17892,0 +17893,0 +17894,0 +17895,0 +17896,0 +17897,0 +17898,0 +17899,0 +17900,0 +17901,0 +17902,0 +17903,0 +17904,0 +17905,0 +17906,0 +17907,0 +17908,0 +17909,0 +17910,0 +17911,0 +17912,0 +17913,0 +17914,0 +17915,0 +17916,0 +17917,0 +17918,0 +17919,0 +17920,0 +17921,0 +17922,0 +17923,0 +17924,0 +17925,0 +17926,0 +17927,0 +17928,0 +17929,0 +17930,0 +17931,0 +17932,0 +17933,0 +17934,0 +17935,0 +17936,0 +17937,0 +17938,0 +17939,0 +17940,0 +17941,0 +17942,0 +17943,0 +17944,0 +17945,0 +17946,0 +17947,0 +17948,0 +17949,0 +17950,0 +17951,0 +17952,0 +17953,0 +17954,0 +17955,0 +17956,0 +17957,0 +17958,0 +17959,0 +17960,0 +17961,0 +17962,0 +17963,0 +17964,0 +17965,0 +17966,0 +17967,0 +17968,0 +17969,0 +17970,0 +17971,0 +17972,0 +17973,0 +17974,0 +17975,0 +17976,0 +17977,0 +17978,0 +17979,0 +17980,0 +17981,0 +17982,0 +17983,0 +17984,0 +17985,0 +17986,0 +17987,0 +17988,0 +17989,0 +17990,0 +17991,0 +17992,0 +17993,0 +17994,0 +17995,0 +17996,0 +17997,0 +17998,0 +17999,0 +18000,0 +18001,0 +18002,0 +18003,0 +18004,0 +18005,0 +18006,0 +18007,0 +18008,0 +18009,0 +18010,0 +18011,0 +18012,0 +18013,0 +18014,0 +18015,0 +18016,0 +18017,0 +18018,0 +18019,0 +18020,0 +18021,0 +18022,0 +18023,0 +18024,0 +18025,0 +18026,0 +18027,0 +18028,0 +18029,0 +18030,0 +18031,0 +18032,0 +18033,0 +18034,0 +18035,0 +18036,0 +18037,0 +18038,0 +18039,0 +18040,0 +18041,0 +18042,0 +18043,0 +18044,0 +18045,0 +18046,0 +18047,0 +18048,0 +18049,0 +18050,0 +18051,0 +18052,0 +18053,0 +18054,0 +18055,0 +18056,0 +18057,0 +18058,0 +18059,0 +18060,0 +18061,0 +18062,0 +18063,0 +18064,0 +18065,0 +18066,0 +18067,0 +18068,0 +18069,0 +18070,0 +18071,0 +18072,0 +18073,0 +18074,0 +18075,0 +18076,0 +18077,0 +18078,0 +18079,0 +18080,0 +18081,0 +18082,0 +18083,0 +18084,0 +18085,0 +18086,0 +18087,0 +18088,0 +18089,0 +18090,0 +18091,0 +18092,0 +18093,0 +18094,0 +18095,0 +18096,0 +18097,0 +18098,0 +18099,0 +18100,0 +18101,0 +18102,0 +18103,0 +18104,0 +18105,0 +18106,0 +18107,0 +18108,0 +18109,0 +18110,0 +18111,0 +18112,0 +18113,0 +18114,0 +18115,0 +18116,0 +18117,0 +18118,0 +18119,0 +18120,0 +18121,0 +18122,0 +18123,0 +18124,0 +18125,0 +18126,0 +18127,0 +18128,0 +18129,0 +18130,0 +18131,0 +18132,0 +18133,0 +18134,0 +18135,0 +18136,0 +18137,0 +18138,0 +18139,0 +18140,0 +18141,0 +18142,0 +18143,0 +18144,0 +18145,0 +18146,0 +18147,0 +18148,0 +18149,0 +18150,0 +18151,0 +18152,0 +18153,0 +18154,0 +18155,0 +18156,0 +18157,0 +18158,0 +18159,0 +18160,0 +18161,0 +18162,0 +18163,0 +18164,0 +18165,0 +18166,0 +18167,0 +18168,0 +18169,0 +18170,0 +18171,0 +18172,0 +18173,0 +18174,0 +18175,0 +18176,0 +18177,0 +18178,0 +18179,0 +18180,0 +18181,0 +18182,0 +18183,0 +18184,0 +18185,0 +18186,0 +18187,0 +18188,0 +18189,0 +18190,0 +18191,0 +18192,0 +18193,0 +18194,0 +18195,0 +18196,0 +18197,0 +18198,0 +18199,0 +18200,0 +18201,0 +18202,0 +18203,0 +18204,0 +18205,0 +18206,0 +18207,0 +18208,0 +18209,0 +18210,0 +18211,0 +18212,0 +18213,0 +18214,0 +18215,0 +18216,0 +18217,0 +18218,0 +18219,0 +18220,0 +18221,0 +18222,0 +18223,0 +18224,0 +18225,0 +18226,0 +18227,0 +18228,0 +18229,0 +18230,0 +18231,0 +18232,0 +18233,0 +18234,0 +18235,0 +18236,0 +18237,0 +18238,0 +18239,0 +18240,0 +18241,0 +18242,0 +18243,0 +18244,0 +18245,0 +18246,0 +18247,0 +18248,0 +18249,0 +18250,0 +18251,0 +18252,0 +18253,0 +18254,0 +18255,0 +18256,0 +18257,0 +18258,0 +18259,0 +18260,0 +18261,0 +18262,0 +18263,0 +18264,0 +18265,0 +18266,0 +18267,0 +18268,0 +18269,0 +18270,0 +18271,0 +18272,0 +18273,0 +18274,0 +18275,0 +18276,0 +18277,0 +18278,0 +18279,0 +18280,0 +18281,0 +18282,0 +18283,0 +18284,0 +18285,0 +18286,0 +18287,0 +18288,0 +18289,0 +18290,0 +18291,0 +18292,0 +18293,0 +18294,0 +18295,0 +18296,0 +18297,0 +18298,0 +18299,0 +18300,0 +18301,0 +18302,0 +18303,0 +18304,0 +18305,0 +18306,0 +18307,0 +18308,0 +18309,0 +18310,0 +18311,0 +18312,0 +18313,0 +18314,0 +18315,0 +18316,0 +18317,0 +18318,0 +18319,0 +18320,0 +18321,0 +18322,0 +18323,0 +18324,0 +18325,0 +18326,0 +18327,0 +18328,0 +18329,0 +18330,0 +18331,0 +18332,0 +18333,0 +18334,0 +18335,0 +18336,0 +18337,0 +18338,0 +18339,0 +18340,0 +18341,0 +18342,0 +18343,0 +18344,0 +18345,0 +18346,0 +18347,0 +18348,0 +18349,0 +18350,0 +18351,0 +18352,0 +18353,0 +18354,0 +18355,0 +18356,0 +18357,0 +18358,0 +18359,0 +18360,0 +18361,0 +18362,0 +18363,0 +18364,0 +18365,0 +18366,0 +18367,0 +18368,0 +18369,0 +18370,0 +18371,0 +18372,0 +18373,0 +18374,0 +18375,0 +18376,0 +18377,0 +18378,0 +18379,0 +18380,0 +18381,0 +18382,0 +18383,0 +18384,0 +18385,0 +18386,0 +18387,0 +18388,0 +18389,0 +18390,0 +18391,0 +18392,0 +18393,0 +18394,0 +18395,0 +18396,0 +18397,0 +18398,0 +18399,0 +18400,0 +18401,0 +18402,0 +18403,0 +18404,0 +18405,0 +18406,0 +18407,0 +18408,0 +18409,0 +18410,0 +18411,0 +18412,0 +18413,0 +18414,0 +18415,0 +18416,0 +18417,0 +18418,0 +18419,0 +18420,0 +18421,0 +18422,0 +18423,0 +18424,0 +18425,0 +18426,0 +18427,0 +18428,0 +18429,0 +18430,0 +18431,0 +18432,0 +18433,0 +18434,0 +18435,0 +18436,0 +18437,0 +18438,0 +18439,0 +18440,0 +18441,0 +18442,0 +18443,0 +18444,0 +18445,0 +18446,0 +18447,0 +18448,0 +18449,0 +18450,0 +18451,0 +18452,0 +18453,0 +18454,0 +18455,0 +18456,0 +18457,0 +18458,0 +18459,0 +18460,0 +18461,0 +18462,0 +18463,0 +18464,0 +18465,0 +18466,0 +18467,0 +18468,0 +18469,0 +18470,0 +18471,0 +18472,0 +18473,0 +18474,0 +18475,0 +18476,0 +18477,0 +18478,0 +18479,0 +18480,0 +18481,0 +18482,0 +18483,0 +18484,0 +18485,0 +18486,0 +18487,0 +18488,0 +18489,0 +18490,0 +18491,0 +18492,0 +18493,0 +18494,0 +18495,0 +18496,0 +18497,0 +18498,0 +18499,0 +18500,0 +18501,0 +18502,0 +18503,0 +18504,0 +18505,0 +18506,0 +18507,0 +18508,0 +18509,0 +18510,0 +18511,0 +18512,0 +18513,0 +18514,0 +18515,0 +18516,0 +18517,0 +18518,0 +18519,0 +18520,0 +18521,0 +18522,0 +18523,0 +18524,0 +18525,0 +18526,0 +18527,0 +18528,0 +18529,0 +18530,0 +18531,0 +18532,0 +18533,0 +18534,0 +18535,0 +18536,0 +18537,0 +18538,0 +18539,0 +18540,0 +18541,0 +18542,0 +18543,0 +18544,0 +18545,0 +18546,0 +18547,0 +18548,0 +18549,0 +18550,0 +18551,0 +18552,0 +18553,0 +18554,0 +18555,0 +18556,0 +18557,0 +18558,0 +18559,0 +18560,0 +18561,0 +18562,0 +18563,0 +18564,0 +18565,0 +18566,0 +18567,0 +18568,0 +18569,0 +18570,0 +18571,0 +18572,0 +18573,0 +18574,0 +18575,0 +18576,0 +18577,0 +18578,0 +18579,0 +18580,0 +18581,0 +18582,0 +18583,0 +18584,0 +18585,0 +18586,0 +18587,0 +18588,0 +18589,0 +18590,0 +18591,0 +18592,0 +18593,0 +18594,0 +18595,0 +18596,0 +18597,0 +18598,0 +18599,0 +18600,0 +18601,0 +18602,0 +18603,0 +18604,0 +18605,0 +18606,0 +18607,0 +18608,0 +18609,0 +18610,0 +18611,0 +18612,0 +18613,0 +18614,0 +18615,0 +18616,0 +18617,0 +18618,0 +18619,0 +18620,0 +18621,0 +18622,0 +18623,0 +18624,0 +18625,0 +18626,0 +18627,0 +18628,0 +18629,0 +18630,0 +18631,0 +18632,0 +18633,0 +18634,0 +18635,0 +18636,0 +18637,0 +18638,0 +18639,0 +18640,0 +18641,0 +18642,0 +18643,0 +18644,0 +18645,0 +18646,0 +18647,0 +18648,0 +18649,0 +18650,0 +18651,0 +18652,0 +18653,0 +18654,0 +18655,0 +18656,0 +18657,0 +18658,0 +18659,0 +18660,0 +18661,0 +18662,0 +18663,0 +18664,0 +18665,0 +18666,0 +18667,0 +18668,0 +18669,0 +18670,0 +18671,0 +18672,0 +18673,0 +18674,0 +18675,0 +18676,0 +18677,0 +18678,0 +18679,0 +18680,0 +18681,0 +18682,0 +18683,0 +18684,0 +18685,0 +18686,0 +18687,0 +18688,0 +18689,0 +18690,0 +18691,0 +18692,0 +18693,0 +18694,0 +18695,0 +18696,0 +18697,0 +18698,0 +18699,0 +18700,0 +18701,0 +18702,0 +18703,0 +18704,0 +18705,0 +18706,0 +18707,0 +18708,0 +18709,0 +18710,0 +18711,0 +18712,0 +18713,0 +18714,0 +18715,0 +18716,0 +18717,0 +18718,0 +18719,0 +18720,0 +18721,0 +18722,0 +18723,0 +18724,0 +18725,0 +18726,0 +18727,0 +18728,0 +18729,0 +18730,0 +18731,0 +18732,0 +18733,0 +18734,0 +18735,0 +18736,0 +18737,0 +18738,0 +18739,0 +18740,0 +18741,0 +18742,0 +18743,0 +18744,0 +18745,0 +18746,0 +18747,0 +18748,0 +18749,0 +18750,0 +18751,0 +18752,0 +18753,0 +18754,0 +18755,0 +18756,0 +18757,0 +18758,0 +18759,0 +18760,0 +18761,0 +18762,0 +18763,0 +18764,0 +18765,0 +18766,0 +18767,0 +18768,0 +18769,0 +18770,0 +18771,0 +18772,0 +18773,0 +18774,0 +18775,0 +18776,0 +18777,0 +18778,0 +18779,0 +18780,0 +18781,0 +18782,0 +18783,0 +18784,0 +18785,0 +18786,0 +18787,0 +18788,0 +18789,0 +18790,0 +18791,0 +18792,0 +18793,0 +18794,0 +18795,0 +18796,0 +18797,0 +18798,0 +18799,0 +18800,0 +18801,0 +18802,0 +18803,0 +18804,0 +18805,0 +18806,0 +18807,0 +18808,0 +18809,0 +18810,0 +18811,0 +18812,0 +18813,0 +18814,0 +18815,0 +18816,0 +18817,0 +18818,0 +18819,0 +18820,0 +18821,0 +18822,0 +18823,0 +18824,0 +18825,0 +18826,0 +18827,0 +18828,0 +18829,0 +18830,0 +18831,0 +18832,0 +18833,0 +18834,0 +18835,0 +18836,0 +18837,0 +18838,0 +18839,0 +18840,0 +18841,0 +18842,0 +18843,0 +18844,0 +18845,0 +18846,0 +18847,0 +18848,0 +18849,0 +18850,0 +18851,0 +18852,0 +18853,0 +18854,0 +18855,0 +18856,0 +18857,0 +18858,0 +18859,0 +18860,0 +18861,0 +18862,0 +18863,0 +18864,0 +18865,0 +18866,0 +18867,0 +18868,0 +18869,0 +18870,0 +18871,0 +18872,0 +18873,0 +18874,0 +18875,0 +18876,0 +18877,0 +18878,0 +18879,0 +18880,0 +18881,0 +18882,0 +18883,0 +18884,0 +18885,0 +18886,0 +18887,0 +18888,0 +18889,0 +18890,0 +18891,0 +18892,0 +18893,0 +18894,0 +18895,0 +18896,0 +18897,0 +18898,0 +18899,0 +18900,0 +18901,0 +18902,0 +18903,0 +18904,0 +18905,0 +18906,0 +18907,0 +18908,0 +18909,0 +18910,0 +18911,0 +18912,0 +18913,0 +18914,0 +18915,0 +18916,0 +18917,0 +18918,0 +18919,0 +18920,0 +18921,0 +18922,0 +18923,0 +18924,0 +18925,0 +18926,0 +18927,0 +18928,0 +18929,0 +18930,0 +18931,0 +18932,0 +18933,0 +18934,0 +18935,0 +18936,0 +18937,0 +18938,0 +18939,0 +18940,0 +18941,0 +18942,0 +18943,0 +18944,0 +18945,0 +18946,0 +18947,0 +18948,0 +18949,0 +18950,0 +18951,0 +18952,0 +18953,0 +18954,0 +18955,0 +18956,0 +18957,0 +18958,0 +18959,0 +18960,0 +18961,0 +18962,0 +18963,0 +18964,0 +18965,0 +18966,0 +18967,0 +18968,0 +18969,0 +18970,0 +18971,0 +18972,0 +18973,0 +18974,0 +18975,0 +18976,0 +18977,0 +18978,0 +18979,0 +18980,0 +18981,0 +18982,0 +18983,0 +18984,0 +18985,0 +18986,0 +18987,0 +18988,0 +18989,0 +18990,0 +18991,0 +18992,0 +18993,0 +18994,0 +18995,0 +18996,0 +18997,0 +18998,0 +18999,0 +19000,0 +19001,0 +19002,0 +19003,0 +19004,0 +19005,0 +19006,0 +19007,0 +19008,0 +19009,0 +19010,0 +19011,0 +19012,0 +19013,0 +19014,0 +19015,0 +19016,0 +19017,0 +19018,0 +19019,0 +19020,0 +19021,0 +19022,0 +19023,0 +19024,0 +19025,0 +19026,0 +19027,0 +19028,0 +19029,0 +19030,0 +19031,0 +19032,0 +19033,0 +19034,0 +19035,0 +19036,0 +19037,0 +19038,0 +19039,0 +19040,0 +19041,0 +19042,0 +19043,0 +19044,0 +19045,0 +19046,0 +19047,0 +19048,0 +19049,0 +19050,0 +19051,0 +19052,0 +19053,0 +19054,0 +19055,0 +19056,0 +19057,0 +19058,0 +19059,0 +19060,0 +19061,0 +19062,0 +19063,0 +19064,0 +19065,0 +19066,0 +19067,0 +19068,0 +19069,0 +19070,0 +19071,0 +19072,0 +19073,0 +19074,0 +19075,0 +19076,0 +19077,0 +19078,0 +19079,0 +19080,0 +19081,0 +19082,0 +19083,0 +19084,0 +19085,0 +19086,0 +19087,0 +19088,0 +19089,0 +19090,0 +19091,0 +19092,0 +19093,0 +19094,0 +19095,0 +19096,0 +19097,0 +19098,0 +19099,0 +19100,0 +19101,0 +19102,0 +19103,0 +19104,0 +19105,0 +19106,0 +19107,0 +19108,0 +19109,0 +19110,0 +19111,0 +19112,0 +19113,0 +19114,0 +19115,0 +19116,0 +19117,0 +19118,0 +19119,0 +19120,0 +19121,0 +19122,0 +19123,0 +19124,0 +19125,0 +19126,0 +19127,0 +19128,0 +19129,0 +19130,0 +19131,0 +19132,0 +19133,0 +19134,0 +19135,0 +19136,0 +19137,0 +19138,0 +19139,0 +19140,0 +19141,0 +19142,0 +19143,0 +19144,0 +19145,0 +19146,0 +19147,0 +19148,0 +19149,0 +19150,0 +19151,0 +19152,0 +19153,0 +19154,0 +19155,0 +19156,0 +19157,0 +19158,0 +19159,0 +19160,0 +19161,0 +19162,0 +19163,0 +19164,0 +19165,0 +19166,0 +19167,0 +19168,0 +19169,0 +19170,0 +19171,0 +19172,0 +19173,0 +19174,0 +19175,0 +19176,0 +19177,0 +19178,0 +19179,0 +19180,0 +19181,0 +19182,0 +19183,0 +19184,0 +19185,0 +19186,0 +19187,0 +19188,0 +19189,0 +19190,0 +19191,0 +19192,0 +19193,0 +19194,0 +19195,0 +19196,0 +19197,0 +19198,0 +19199,0 +19200,0 +19201,0 +19202,0 +19203,0 +19204,0 +19205,0 +19206,0 +19207,0 +19208,0 +19209,0 +19210,0 +19211,0 +19212,0 +19213,0 +19214,0 +19215,0 +19216,0 +19217,0 +19218,0 +19219,0 +19220,0 +19221,0 +19222,0 +19223,0 +19224,0 +19225,0 +19226,0 +19227,0 +19228,0 +19229,0 +19230,0 +19231,0 +19232,0 +19233,0 +19234,0 +19235,0 +19236,0 +19237,0 +19238,0 +19239,0 +19240,0 +19241,0 +19242,0 +19243,0 +19244,0 +19245,0 +19246,0 +19247,0 +19248,0 +19249,0 +19250,0 +19251,0 +19252,0 +19253,0 +19254,0 +19255,0 +19256,0 +19257,0 +19258,0 +19259,0 +19260,0 +19261,0 +19262,0 +19263,0 +19264,0 +19265,0 +19266,0 +19267,0 +19268,0 +19269,0 +19270,0 +19271,0 +19272,0 +19273,0 +19274,0 +19275,0 +19276,0 +19277,0 +19278,0 +19279,0 +19280,0 +19281,0 +19282,0 +19283,0 +19284,0 +19285,0 +19286,0 +19287,0 +19288,0 +19289,0 +19290,0 +19291,0 +19292,0 +19293,0 +19294,0 +19295,0 +19296,0 +19297,0 +19298,0 +19299,0 +19300,0 +19301,0 +19302,0 +19303,0 +19304,0 +19305,0 +19306,0 +19307,0 +19308,0 +19309,0 +19310,0 +19311,0 +19312,0 +19313,0 +19314,0 +19315,0 +19316,0 +19317,0 +19318,0 +19319,0 +19320,0 +19321,0 +19322,0 +19323,0 +19324,0 +19325,0 +19326,0 +19327,0 +19328,0 +19329,0 +19330,0 +19331,0 +19332,0 +19333,0 +19334,0 +19335,0 +19336,0 +19337,0 +19338,0 +19339,0 +19340,0 +19341,0 +19342,0 +19343,0 +19344,0 +19345,0 +19346,0 +19347,0 +19348,0 +19349,0 +19350,0 +19351,0 +19352,0 +19353,0 +19354,0 +19355,0 +19356,0 +19357,0 +19358,0 +19359,0 +19360,0 +19361,0 +19362,0 +19363,0 +19364,0 +19365,0 +19366,0 +19367,0 +19368,0 +19369,0 +19370,0 +19371,0 +19372,0 +19373,0 +19374,0 +19375,0 +19376,0 +19377,0 +19378,0 +19379,0 +19380,0 +19381,0 +19382,0 +19383,0 +19384,0 +19385,0 +19386,0 +19387,0 +19388,0 +19389,0 +19390,0 +19391,0 +19392,0 +19393,0 +19394,0 +19395,0 +19396,0 +19397,0 +19398,0 +19399,0 +19400,0 +19401,0 +19402,0 +19403,0 +19404,0 +19405,0 +19406,0 +19407,0 +19408,0 +19409,0 +19410,0 +19411,0 +19412,0 +19413,0 +19414,0 +19415,0 +19416,0 +19417,0 +19418,0 +19419,0 +19420,0 +19421,0 +19422,0 +19423,0 +19424,0 +19425,0 +19426,0 +19427,0 +19428,0 +19429,0 +19430,0 +19431,0 +19432,0 +19433,0 +19434,0 +19435,0 +19436,0 +19437,0 +19438,0 +19439,0 +19440,0 +19441,0 +19442,0 +19443,0 +19444,0 +19445,0 +19446,0 +19447,0 +19448,0 +19449,0 +19450,0 +19451,0 +19452,0 +19453,0 +19454,0 +19455,0 +19456,0 +19457,0 +19458,0 +19459,0 +19460,0 +19461,0 +19462,0 +19463,0 +19464,0 +19465,0 +19466,0 +19467,0 +19468,0 +19469,0 +19470,0 +19471,0 +19472,0 +19473,0 +19474,0 +19475,0 +19476,0 +19477,0 +19478,0 +19479,0 +19480,0 +19481,0 +19482,0 +19483,0 +19484,0 +19485,0 +19486,0 +19487,0 +19488,0 +19489,0 +19490,0 +19491,0 +19492,0 +19493,0 +19494,0 +19495,0 +19496,0 +19497,0 +19498,0 +19499,0 +19500,0 +19501,0 +19502,0 +19503,0 +19504,0 +19505,0 +19506,0 +19507,0 +19508,0 +19509,0 +19510,0 +19511,0 +19512,0 +19513,0 +19514,0 +19515,0 +19516,0 +19517,0 +19518,0 +19519,0 +19520,0 +19521,0 +19522,0 +19523,0 +19524,0 +19525,0 +19526,0 +19527,0 +19528,0 +19529,0 +19530,0 +19531,0 +19532,0 +19533,0 +19534,0 +19535,0 +19536,0 +19537,0 +19538,0 +19539,0 +19540,0 +19541,0 +19542,0 +19543,0 +19544,0 +19545,0 +19546,0 +19547,0 +19548,0 +19549,0 +19550,0 +19551,0 +19552,0 +19553,0 +19554,0 +19555,0 +19556,0 +19557,0 +19558,0 +19559,0 +19560,0 +19561,0 +19562,0 +19563,0 +19564,0 +19565,0 +19566,0 +19567,0 +19568,0 +19569,0 +19570,0 +19571,0 +19572,0 +19573,0 +19574,0 +19575,0 +19576,0 +19577,0 +19578,0 +19579,0 +19580,0 +19581,0 +19582,0 +19583,0 +19584,0 +19585,0 +19586,0 +19587,0 +19588,0 +19589,0 +19590,0 +19591,0 +19592,0 +19593,0 +19594,0 +19595,0 +19596,0 +19597,0 +19598,0 +19599,0 +19600,0 +19601,0 +19602,0 +19603,0 +19604,0 +19605,0 +19606,0 +19607,0 +19608,0 +19609,0 +19610,0 +19611,0 +19612,0 +19613,0 +19614,0 +19615,0 +19616,0 +19617,0 +19618,0 +19619,0 +19620,0 +19621,0 +19622,0 +19623,0 +19624,0 +19625,0 +19626,0 +19627,0 +19628,0 +19629,0 +19630,0 +19631,0 +19632,0 +19633,0 +19634,0 +19635,0 +19636,0 +19637,0 +19638,0 +19639,0 +19640,0 +19641,0 +19642,0 +19643,0 +19644,0 +19645,0 +19646,0 +19647,0 +19648,0 +19649,0 +19650,0 +19651,0 +19652,0 +19653,0 +19654,0 +19655,0 +19656,0 +19657,0 +19658,0 +19659,0 +19660,0 +19661,0 +19662,0 +19663,0 +19664,0 +19665,0 +19666,0 +19667,0 +19668,0 +19669,0 +19670,0 +19671,0 +19672,0 +19673,0 +19674,0 +19675,0 +19676,0 +19677,0 +19678,0 +19679,0 +19680,0 +19681,0 +19682,0 +19683,0 +19684,0 +19685,0 +19686,0 +19687,0 +19688,0 +19689,0 +19690,0 +19691,0 +19692,0 +19693,0 +19694,0 +19695,0 +19696,0 +19697,0 +19698,0 +19699,0 +19700,0 +19701,0 +19702,0 +19703,0 +19704,0 +19705,0 +19706,0 +19707,0 +19708,0 +19709,0 +19710,0 +19711,0 +19712,0 +19713,0 +19714,0 +19715,0 +19716,0 +19717,0 +19718,0 +19719,0 +19720,0 +19721,0 +19722,0 +19723,0 +19724,0 +19725,0 +19726,0 +19727,0 +19728,0 +19729,0 +19730,0 +19731,0 +19732,0 +19733,0 +19734,0 +19735,0 +19736,0 +19737,0 +19738,0 +19739,0 +19740,0 +19741,0 +19742,0 +19743,0 +19744,0 +19745,0 +19746,0 +19747,0 +19748,0 +19749,0 +19750,0 +19751,0 +19752,0 +19753,0 +19754,0 +19755,0 +19756,0 +19757,0 +19758,0 +19759,0 +19760,0 +19761,0 +19762,0 +19763,0 +19764,0 +19765,0 +19766,0 +19767,0 +19768,0 +19769,0 +19770,0 +19771,0 +19772,0 +19773,0 +19774,0 +19775,0 +19776,0 +19777,0 +19778,0 +19779,0 +19780,0 +19781,0 +19782,0 +19783,0 +19784,0 +19785,0 +19786,0 +19787,0 +19788,0 +19789,0 +19790,0 +19791,0 +19792,0 +19793,0 +19794,0 +19795,0 +19796,0 +19797,0 +19798,0 +19799,0 +19800,0 +19801,0 +19802,0 +19803,0 +19804,0 +19805,0 +19806,0 +19807,0 +19808,0 +19809,0 +19810,0 +19811,0 +19812,0 +19813,0 +19814,0 +19815,0 +19816,0 +19817,0 +19818,0 +19819,0 +19820,0 +19821,0 +19822,0 +19823,0 +19824,0 +19825,0 +19826,0 +19827,0 +19828,0 +19829,0 +19830,0 +19831,0 +19832,0 +19833,0 +19834,0 +19835,0 +19836,0 +19837,0 +19838,0 +19839,0 +19840,0 +19841,0 +19842,0 +19843,0 +19844,0 +19845,0 +19846,0 +19847,0 +19848,0 +19849,0 +19850,0 +19851,0 +19852,0 +19853,0 +19854,0 +19855,0 +19856,0 +19857,0 +19858,0 +19859,0 +19860,0 +19861,0 +19862,0 +19863,0 +19864,0 +19865,0 +19866,0 +19867,0 +19868,0 +19869,0 +19870,0 +19871,0 +19872,0 +19873,0 +19874,0 +19875,0 +19876,0 +19877,0 +19878,0 +19879,0 +19880,0 +19881,0 +19882,0 +19883,0 +19884,0 +19885,0 +19886,0 +19887,0 +19888,0 +19889,0 +19890,0 +19891,0 +19892,0 +19893,0 +19894,0 +19895,0 +19896,0 +19897,0 +19898,0 +19899,0 +19900,0 +19901,0 +19902,0 +19903,0 +19904,0 +19905,0 +19906,0 +19907,0 +19908,0 +19909,0 +19910,0 +19911,0 +19912,0 +19913,0 +19914,0 +19915,0 +19916,0 +19917,0 +19918,0 +19919,0 +19920,0 +19921,0 +19922,0 +19923,0 +19924,0 +19925,0 +19926,0 +19927,0 +19928,0 +19929,0 +19930,0 +19931,0 +19932,0 +19933,0 +19934,0 +19935,0 +19936,0 +19937,0 +19938,0 +19939,0 +19940,0 +19941,0 +19942,0 +19943,0 +19944,0 +19945,0 +19946,0 +19947,0 +19948,0 +19949,0 +19950,0 +19951,0 +19952,0 +19953,0 +19954,0 +19955,0 +19956,0 +19957,0 +19958,0 +19959,0 +19960,0 +19961,0 +19962,0 +19963,0 +19964,0 +19965,0 +19966,0 +19967,0 +19968,0 +19969,0 +19970,0 +19971,0 +19972,0 +19973,0 +19974,0 +19975,0 +19976,0 +19977,0 +19978,0 +19979,0 +19980,0 +19981,0 +19982,0 +19983,0 +19984,0 +19985,0 +19986,0 +19987,0 +19988,0 +19989,0 +19990,0 +19991,0 +19992,0 +19993,0 +19994,0 +19995,0 +19996,0 +19997,0 +19998,0 +19999,0 +20000,0 +20001,0 +20002,0 +20003,0 +20004,0 +20005,0 +20006,0 +20007,0 +20008,0 +20009,0 +20010,0 +20011,0 +20012,0 +20013,0 +20014,0 +20015,0 +20016,0 +20017,0 +20018,0 +20019,0 +20020,0 +20021,0 +20022,0 +20023,0 +20024,0 +20025,0 +20026,0 +20027,0 +20028,0 +20029,0 +20030,0 +20031,0 +20032,0 +20033,0 +20034,0 +20035,0 +20036,0 +20037,0 +20038,0 +20039,0 +20040,0 +20041,0 +20042,0 +20043,0 +20044,0 +20045,0 +20046,0 +20047,0 +20048,0 +20049,0 +20050,0 +20051,0 +20052,0 +20053,0 +20054,0 +20055,0 +20056,0 +20057,0 +20058,0 +20059,0 +20060,0 +20061,0 +20062,0 +20063,0 +20064,0 +20065,0 +20066,0 +20067,0 +20068,0 +20069,0 +20070,0 +20071,0 +20072,0 +20073,0 +20074,0 +20075,0 +20076,0 +20077,0 +20078,0 +20079,0 +20080,0 +20081,0 +20082,0 +20083,0 +20084,0 +20085,0 +20086,0 +20087,0 +20088,0 +20089,0 +20090,0 +20091,0 +20092,0 +20093,0 +20094,0 +20095,0 +20096,0 +20097,0 +20098,0 +20099,0 +20100,0 +20101,0 +20102,0 +20103,0 +20104,0 +20105,0 +20106,0 +20107,0 +20108,0 +20109,0 +20110,0 +20111,0 +20112,0 +20113,0 +20114,0 +20115,0 +20116,0 +20117,0 +20118,0 +20119,0 +20120,0 +20121,0 +20122,0 +20123,0 +20124,0 +20125,0 +20126,0 +20127,0 +20128,0 +20129,0 +20130,0 +20131,0 +20132,0 +20133,0 +20134,0 +20135,0 +20136,0 +20137,0 +20138,0 +20139,0 +20140,0 +20141,0 +20142,0 +20143,0 +20144,0 +20145,0 +20146,0 +20147,0 +20148,0 +20149,0 +20150,0 +20151,0 +20152,0 +20153,0 +20154,0 +20155,0 +20156,0 +20157,0 +20158,0 +20159,0 +20160,0 +20161,0 +20162,0 +20163,0 +20164,0 +20165,0 +20166,0 +20167,0 +20168,0 +20169,0 +20170,0 +20171,0 +20172,0 +20173,0 +20174,0 +20175,0 +20176,0 +20177,0 +20178,0 +20179,0 +20180,0 +20181,0 +20182,0 +20183,0 +20184,0 +20185,0 +20186,0 +20187,0 +20188,0 +20189,0 +20190,0 +20191,0 +20192,0 +20193,0 +20194,0 +20195,0 +20196,0 +20197,0 +20198,0 +20199,0 +20200,0 +20201,0 +20202,0 +20203,0 +20204,0 +20205,0 +20206,0 +20207,0 +20208,0 +20209,0 +20210,0 +20211,0 +20212,0 +20213,0 +20214,0 +20215,0 +20216,0 +20217,0 +20218,0 +20219,0 +20220,0 +20221,0 +20222,0 +20223,0 +20224,0 +20225,0 +20226,0 +20227,0 +20228,0 +20229,0 +20230,0 +20231,0 +20232,0 +20233,0 +20234,0 +20235,0 +20236,0 +20237,0 +20238,0 +20239,0 +20240,0 +20241,0 +20242,0 +20243,0 +20244,0 +20245,0 +20246,0 +20247,0 +20248,0 +20249,0 +20250,0 +20251,0 +20252,0 +20253,0 +20254,0 +20255,0 +20256,0 +20257,0 +20258,0 +20259,0 +20260,0 +20261,0 +20262,0 +20263,0 +20264,0 +20265,0 +20266,0 +20267,0 +20268,0 +20269,0 +20270,0 +20271,0 +20272,0 +20273,0 +20274,0 +20275,0 +20276,0 +20277,0 +20278,0 +20279,0 +20280,0 +20281,0 +20282,0 +20283,0 +20284,0 +20285,0 +20286,0 +20287,0 +20288,0 +20289,0 +20290,0 +20291,0 +20292,0 +20293,0 +20294,0 +20295,0 +20296,0 +20297,0 +20298,0 +20299,0 +20300,0 +20301,0 +20302,0 +20303,0 +20304,0 +20305,0 +20306,0 +20307,0 +20308,0 +20309,0 +20310,0 +20311,0 +20312,0 +20313,0 +20314,0 +20315,0 +20316,0 +20317,0 +20318,0 +20319,0 +20320,0 +20321,0 +20322,0 +20323,0 +20324,0 +20325,0 +20326,0 +20327,0 +20328,0 +20329,0 +20330,0 +20331,0 +20332,0 +20333,0 +20334,0 +20335,0 +20336,0 +20337,0 +20338,0 +20339,0 +20340,0 +20341,0 +20342,0 +20343,0 +20344,0 +20345,0 +20346,0 +20347,0 +20348,0 +20349,0 +20350,0 +20351,0 +20352,0 +20353,0 +20354,0 +20355,0 +20356,0 +20357,0 +20358,0 +20359,0 +20360,0 +20361,0 +20362,0 +20363,0 +20364,0 +20365,0 +20366,0 +20367,0 +20368,0 +20369,0 +20370,0 +20371,0 +20372,0 +20373,0 +20374,0 +20375,0 +20376,0 +20377,0 +20378,0 +20379,0 +20380,0 +20381,0 +20382,0 +20383,0 +20384,0 +20385,0 +20386,0 +20387,0 +20388,0 +20389,0 +20390,0 +20391,0 +20392,0 +20393,0 +20394,0 +20395,0 +20396,0 +20397,0 +20398,0 +20399,0 +20400,0 +20401,0 +20402,0 +20403,0 +20404,0 +20405,0 +20406,0 +20407,0 +20408,0 +20409,0 +20410,0 +20411,0 +20412,0 +20413,0 +20414,0 +20415,0 +20416,0 +20417,0 +20418,0 +20419,0 +20420,0 +20421,0 +20422,0 +20423,0 +20424,0 +20425,0 +20426,0 +20427,0 +20428,0 +20429,0 +20430,0 +20431,0 +20432,0 +20433,0 +20434,0 +20435,0 +20436,0 +20437,0 +20438,0 +20439,0 +20440,0 +20441,0 +20442,0 +20443,0 +20444,0 +20445,0 +20446,0 +20447,0 +20448,0 +20449,0 +20450,0 +20451,0 +20452,0 +20453,0 +20454,0 +20455,0 +20456,0 +20457,0 +20458,0 +20459,0 +20460,0 +20461,0 +20462,0 +20463,0 +20464,0 +20465,0 +20466,0 +20467,0 +20468,0 +20469,0 +20470,0 +20471,0 +20472,0 +20473,0 +20474,0 +20475,0 +20476,0 +20477,0 +20478,0 +20479,0 +20480,0 +20481,0 +20482,0 +20483,0 +20484,0 +20485,0 +20486,0 +20487,0 +20488,0 +20489,0 +20490,0 +20491,0 +20492,0 +20493,0 +20494,0 +20495,0 +20496,0 +20497,0 +20498,0 +20499,0 +20500,0 +20501,0 +20502,0 +20503,0 +20504,0 +20505,0 +20506,0 +20507,0 +20508,0 +20509,0 +20510,0 +20511,0 +20512,0 +20513,0 +20514,0 +20515,0 +20516,0 +20517,0 +20518,0 +20519,0 +20520,0 +20521,0 +20522,0 +20523,0 +20524,0 +20525,0 +20526,0 +20527,0 +20528,0 +20529,0 +20530,0 +20531,0 +20532,0 +20533,0 +20534,0 +20535,0 +20536,0 +20537,0 +20538,0 +20539,0 +20540,0 +20541,0 +20542,0 +20543,0 +20544,0 +20545,0 +20546,0 +20547,0 +20548,0 +20549,0 +20550,0 +20551,0 +20552,0 +20553,0 +20554,0 +20555,0 +20556,0 +20557,0 +20558,0 +20559,0 +20560,0 +20561,0 +20562,0 +20563,0 +20564,0 +20565,0 +20566,0 +20567,0 +20568,0 +20569,0 +20570,0 +20571,0 +20572,0 +20573,0 +20574,0 +20575,0 +20576,0 +20577,0 +20578,0 +20579,0 +20580,0 +20581,0 +20582,0 +20583,0 +20584,0 +20585,0 +20586,0 +20587,0 +20588,0 +20589,0 +20590,0 +20591,0 +20592,0 +20593,0 +20594,0 +20595,0 +20596,0 +20597,0 +20598,0 +20599,0 +20600,0 +20601,0 +20602,0 +20603,0 +20604,0 +20605,0 +20606,0 +20607,0 +20608,0 +20609,0 +20610,0 +20611,0 +20612,0 +20613,0 +20614,0 +20615,0 +20616,0 +20617,0 +20618,0 +20619,0 +20620,0 +20621,0 +20622,0 +20623,0 +20624,0 +20625,0 +20626,0 +20627,0 +20628,0 +20629,0 +20630,0 +20631,0 +20632,0 +20633,0 +20634,0 +20635,0 +20636,0 +20637,0 +20638,0 +20639,0 +20640,0 +20641,0 +20642,0 +20643,0 +20644,0 +20645,0 +20646,0 +20647,0 +20648,0 +20649,0 +20650,0 +20651,0 +20652,0 +20653,0 +20654,0 +20655,0 +20656,0 +20657,0 +20658,0 +20659,0 +20660,0 +20661,0 +20662,0 +20663,0 +20664,0 +20665,0 +20666,0 +20667,0 +20668,0 +20669,0 +20670,0 +20671,0 +20672,0 +20673,0 +20674,0 +20675,0 +20676,0 +20677,0 +20678,0 +20679,0 +20680,0 +20681,0 +20682,0 +20683,0 +20684,0 +20685,0 +20686,0 +20687,0 +20688,0 +20689,0 +20690,0 +20691,0 +20692,0 +20693,0 +20694,0 +20695,0 +20696,0 +20697,0 +20698,0 +20699,0 +20700,0 +20701,0 +20702,0 +20703,0 +20704,0 +20705,0 +20706,0 +20707,0 +20708,0 +20709,0 +20710,0 +20711,0 +20712,0 +20713,0 +20714,0 +20715,0 +20716,0 +20717,0 +20718,0 +20719,0 +20720,0 +20721,0 +20722,0 +20723,0 +20724,0 +20725,0 +20726,0 +20727,0 +20728,0 +20729,0 +20730,0 +20731,0 +20732,0 +20733,0 +20734,0 +20735,0 +20736,0 +20737,0 +20738,0 +20739,0 +20740,0 +20741,0 +20742,0 +20743,0 +20744,0 +20745,0 +20746,0 +20747,0 +20748,0 +20749,0 +20750,0 +20751,0 +20752,0 +20753,0 +20754,0 +20755,0 +20756,0 +20757,0 +20758,0 +20759,0 +20760,0 +20761,0 +20762,0 +20763,0 +20764,0 +20765,0 +20766,0 +20767,0 +20768,0 +20769,0 +20770,0 +20771,0 +20772,0 +20773,0 +20774,0 +20775,0 +20776,0 +20777,0 +20778,0 +20779,0 +20780,0 +20781,0 +20782,0 +20783,0 +20784,0 +20785,0 +20786,0 +20787,0 +20788,0 +20789,0 +20790,0 +20791,0 +20792,0 +20793,0 +20794,0 +20795,0 +20796,0 +20797,0 +20798,0 +20799,0 +20800,0 +20801,0 +20802,0 +20803,0 +20804,0 +20805,0 +20806,0 +20807,0 +20808,0 +20809,0 +20810,0 +20811,0 +20812,0 +20813,0 +20814,0 +20815,0 +20816,0 +20817,0 +20818,0 +20819,0 +20820,0 +20821,0 +20822,0 +20823,0 +20824,0 +20825,0 +20826,0 +20827,0 +20828,0 +20829,0 +20830,0 +20831,0 +20832,0 +20833,0 +20834,0 +20835,0 +20836,0 +20837,0 +20838,0 +20839,0 +20840,0 +20841,0 +20842,0 +20843,0 +20844,0 +20845,0 +20846,0 +20847,0 +20848,0 +20849,0 +20850,0 +20851,0 +20852,0 +20853,0 +20854,0 +20855,0 +20856,0 +20857,0 +20858,0 +20859,0 +20860,0 +20861,0 +20862,0 +20863,0 +20864,0 +20865,0 +20866,0 +20867,0 +20868,0 +20869,0 +20870,0 +20871,0 +20872,0 +20873,0 +20874,0 +20875,0 +20876,0 +20877,0 +20878,0 +20879,0 +20880,0 +20881,0 +20882,0 +20883,0 +20884,0 +20885,0 +20886,0 +20887,0 +20888,0 +20889,0 +20890,0 +20891,0 +20892,0 +20893,0 +20894,0 +20895,0 +20896,0 +20897,0 +20898,0 +20899,0 +20900,0 +20901,0 +20902,0 +20903,0 +20904,0 +20905,0 +20906,0 +20907,0 +20908,0 +20909,0 +20910,0 +20911,0 +20912,0 +20913,0 +20914,0 +20915,0 +20916,0 +20917,0 +20918,0 +20919,0 +20920,0 +20921,0 +20922,0 +20923,0 +20924,0 +20925,0 +20926,0 +20927,0 +20928,0 +20929,0 +20930,0 +20931,0 +20932,0 +20933,0 +20934,0 +20935,0 +20936,0 +20937,0 +20938,0 +20939,0 +20940,0 +20941,0 +20942,0 +20943,0 +20944,0 +20945,0 +20946,0 +20947,0 +20948,0 +20949,0 +20950,0 +20951,0 +20952,0 +20953,0 +20954,0 +20955,0 +20956,0 +20957,0 +20958,0 +20959,0 +20960,0 +20961,0 +20962,0 +20963,0 +20964,0 +20965,0 +20966,0 +20967,0 +20968,0 +20969,0 +20970,0 +20971,0 +20972,0 +20973,0 +20974,0 +20975,0 +20976,0 +20977,0 +20978,0 +20979,0 +20980,0 +20981,0 +20982,0 +20983,0 +20984,0 +20985,0 +20986,0 +20987,0 +20988,0 +20989,0 +20990,0 +20991,0 +20992,0 +20993,0 +20994,0 +20995,0 +20996,0 +20997,0 +20998,0 +20999,0 +21000,0 +21001,0 +21002,0 +21003,0 +21004,0 +21005,0 +21006,0 +21007,0 +21008,0 +21009,0 +21010,0 +21011,0 +21012,0 +21013,0 +21014,0 +21015,0 +21016,0 +21017,0 +21018,0 +21019,0 +21020,0 +21021,0 +21022,0 +21023,0 +21024,0 +21025,0 +21026,0 +21027,0 +21028,0 +21029,0 +21030,0 +21031,0 +21032,0 +21033,0 +21034,0 +21035,0 +21036,0 +21037,0 +21038,0 +21039,0 +21040,0 +21041,0 +21042,0 +21043,0 +21044,0 +21045,0 +21046,0 +21047,0 +21048,0 +21049,0 +21050,0 +21051,0 +21052,0 +21053,0 +21054,0 +21055,0 +21056,0 +21057,0 +21058,0 +21059,0 +21060,0 +21061,0 +21062,0 +21063,0 +21064,0 +21065,0 +21066,0 +21067,0 +21068,0 +21069,0 +21070,0 +21071,0 +21072,0 +21073,0 +21074,0 +21075,0 +21076,0 +21077,0 +21078,0 +21079,0 +21080,0 +21081,0 +21082,0 +21083,0 +21084,0 +21085,0 +21086,0 +21087,0 +21088,0 +21089,0 +21090,0 +21091,0 +21092,0 +21093,0 +21094,0 +21095,0 +21096,0 +21097,0 +21098,0 +21099,0 +21100,0 +21101,0 +21102,0 +21103,0 +21104,0 +21105,0 +21106,0 +21107,0 +21108,0 +21109,0 +21110,0 +21111,0 +21112,0 +21113,0 +21114,0 +21115,0 +21116,0 +21117,0 +21118,0 +21119,0 +21120,0 +21121,0 +21122,0 +21123,0 +21124,0 +21125,0 +21126,0 +21127,0 +21128,0 +21129,0 +21130,0 +21131,0 +21132,0 +21133,0 +21134,0 +21135,0 +21136,0 +21137,0 +21138,0 +21139,0 +21140,0 +21141,0 +21142,0 +21143,0 +21144,0 +21145,0 +21146,0 +21147,0 +21148,0 +21149,0 +21150,0 +21151,0 +21152,0 +21153,0 +21154,0 +21155,0 +21156,0 +21157,0 +21158,0 +21159,0 +21160,0 +21161,0 +21162,0 +21163,0 +21164,0 +21165,0 +21166,0 +21167,0 +21168,0 +21169,0 +21170,0 +21171,0 +21172,0 +21173,0 +21174,0 +21175,0 +21176,0 +21177,0 +21178,0 +21179,0 +21180,0 +21181,0 +21182,0 +21183,0 +21184,0 +21185,0 +21186,0 +21187,0 +21188,0 +21189,0 +21190,0 +21191,0 +21192,0 +21193,0 +21194,0 +21195,0 +21196,0 +21197,0 +21198,0 +21199,0 +21200,0 +21201,0 +21202,0 +21203,0 +21204,0 +21205,0 +21206,0 +21207,0 +21208,0 +21209,0 +21210,0 +21211,0 +21212,0 +21213,0 +21214,0 +21215,0 +21216,0 +21217,0 +21218,0 +21219,0 +21220,0 +21221,0 +21222,0 +21223,0 +21224,0 +21225,0 +21226,0 +21227,0 +21228,0 +21229,0 +21230,0 +21231,0 +21232,0 +21233,0 +21234,0 +21235,0 +21236,0 +21237,0 +21238,0 +21239,0 +21240,0 +21241,0 +21242,0 +21243,0 +21244,0 +21245,0 +21246,0 +21247,0 +21248,0 +21249,0 +21250,0 +21251,0 +21252,0 +21253,0 +21254,0 +21255,0 +21256,0 +21257,0 +21258,0 +21259,0 +21260,0 +21261,0 +21262,0 +21263,0 +21264,0 +21265,0 +21266,0 +21267,0 +21268,0 +21269,0 +21270,0 +21271,0 +21272,0 +21273,0 +21274,0 +21275,0 +21276,0 +21277,0 +21278,0 +21279,0 +21280,0 +21281,0 +21282,0 +21283,0 +21284,0 +21285,0 +21286,0 +21287,0 +21288,0 +21289,0 +21290,0 +21291,0 +21292,0 +21293,0 +21294,0 +21295,0 +21296,0 +21297,0 +21298,0 +21299,0 +21300,0 +21301,0 +21302,0 +21303,0 +21304,0 +21305,0 +21306,0 +21307,0 +21308,0 +21309,0 +21310,0 +21311,0 +21312,0 +21313,0 +21314,0 +21315,0 +21316,0 +21317,0 +21318,0 +21319,0 +21320,0 +21321,0 +21322,0 +21323,0 +21324,0 +21325,0 +21326,0 +21327,0 +21328,0 +21329,0 +21330,0 +21331,0 +21332,0 +21333,0 +21334,0 +21335,0 +21336,0 +21337,0 +21338,0 +21339,0 +21340,0 +21341,0 +21342,0 +21343,0 +21344,0 +21345,0 +21346,0 +21347,0 +21348,0 +21349,0 +21350,0 +21351,0 +21352,0 +21353,0 +21354,0 +21355,0 +21356,0 +21357,0 +21358,0 +21359,0 +21360,0 +21361,0 +21362,0 +21363,0 +21364,0 +21365,0 +21366,0 +21367,0 +21368,0 +21369,0 +21370,0 +21371,0 +21372,0 +21373,0 +21374,0 +21375,0 +21376,0 +21377,0 +21378,0 +21379,0 +21380,0 +21381,0 +21382,0 +21383,0 +21384,0 +21385,0 +21386,0 +21387,0 +21388,0 +21389,0 +21390,0 +21391,0 +21392,0 +21393,0 +21394,0 +21395,0 +21396,0 +21397,0 +21398,0 +21399,0 +21400,0 +21401,0 +21402,0 +21403,0 +21404,0 +21405,0 +21406,0 +21407,0 +21408,0 +21409,0 +21410,0 +21411,0 +21412,0 +21413,0 +21414,0 +21415,0 +21416,0 +21417,0 +21418,0 +21419,0 +21420,0 +21421,0 +21422,0 +21423,0 +21424,0 +21425,0 +21426,0 +21427,0 +21428,0 +21429,0 +21430,0 +21431,0 +21432,0 +21433,0 +21434,0 +21435,0 +21436,0 +21437,0 +21438,0 +21439,0 +21440,0 +21441,0 +21442,0 +21443,0 +21444,0 +21445,0 +21446,0 +21447,0 +21448,0 +21449,0 +21450,0 +21451,0 +21452,0 +21453,0 +21454,0 +21455,0 +21456,0 +21457,0 +21458,0 +21459,0 +21460,0 +21461,0 +21462,0 +21463,0 +21464,0 +21465,0 +21466,0 +21467,0 +21468,0 +21469,0 +21470,0 +21471,0 +21472,0 +21473,0 +21474,0 +21475,0 +21476,0 +21477,0 +21478,0 +21479,0 +21480,0 +21481,0 +21482,0 +21483,0 +21484,0 +21485,0 +21486,0 +21487,0 +21488,0 +21489,0 +21490,0 +21491,0 +21492,0 +21493,0 +21494,0 +21495,0 +21496,0 +21497,0 +21498,0 +21499,0 +21500,0 +21501,0 +21502,0 +21503,0 +21504,0 +21505,0 +21506,0 +21507,0 +21508,0 +21509,0 +21510,0 +21511,0 +21512,0 +21513,0 +21514,0 +21515,0 +21516,0 +21517,0 +21518,0 +21519,0 +21520,0 +21521,0 +21522,0 +21523,0 +21524,0 +21525,0 +21526,0 +21527,0 +21528,0 +21529,0 +21530,0 +21531,0 +21532,0 +21533,0 +21534,0 +21535,0 +21536,0 +21537,0 +21538,0 +21539,0 +21540,0 +21541,0 +21542,0 +21543,0 +21544,0 +21545,0 +21546,0 +21547,0 +21548,0 +21549,0 +21550,0 +21551,0 +21552,0 +21553,0 +21554,0 +21555,0 +21556,0 +21557,0 +21558,0 +21559,0 +21560,0 +21561,0 +21562,0 +21563,0 +21564,0 +21565,0 +21566,0 +21567,0 +21568,0 +21569,0 +21570,0 +21571,0 +21572,0 +21573,0 +21574,0 +21575,0 +21576,0 +21577,0 +21578,0 +21579,0 +21580,0 +21581,0 +21582,0 +21583,0 +21584,0 +21585,0 +21586,0 +21587,0 +21588,0 +21589,0 +21590,0 +21591,0 +21592,0 +21593,0 +21594,0 +21595,0 +21596,0 +21597,0 +21598,0 +21599,0 +21600,0 +21601,0 +21602,0 +21603,0 +21604,0 +21605,0 +21606,0 +21607,0 +21608,0 +21609,0 +21610,0 +21611,0 +21612,0 +21613,0 +21614,0 +21615,0 +21616,0 +21617,0 +21618,0 +21619,0 +21620,0 +21621,0 +21622,0 +21623,0 +21624,0 +21625,0 +21626,0 +21627,0 +21628,0 +21629,0 +21630,0 +21631,0 +21632,0 +21633,0 +21634,0 +21635,0 +21636,0 +21637,0 +21638,0 +21639,0 +21640,0 +21641,0 +21642,0 +21643,0 +21644,0 +21645,0 +21646,0 +21647,0 +21648,0 +21649,0 +21650,0 +21651,0 +21652,0 +21653,0 +21654,0 +21655,0 +21656,0 +21657,0 +21658,0 +21659,0 +21660,0 +21661,0 +21662,0 +21663,0 +21664,0 +21665,0 +21666,0 +21667,0 +21668,0 +21669,0 +21670,0 +21671,0 +21672,0 +21673,0 +21674,0 +21675,0 +21676,0 +21677,0 +21678,0 +21679,0 +21680,0 +21681,0 +21682,0 +21683,0 +21684,0 +21685,0 +21686,0 +21687,0 +21688,0 +21689,0 +21690,0 +21691,0 +21692,0 +21693,0 +21694,0 +21695,0 +21696,0 +21697,0 +21698,0 +21699,0 +21700,0 +21701,0 +21702,0 +21703,0 +21704,0 +21705,0 +21706,0 +21707,0 +21708,0 +21709,0 +21710,0 +21711,0 +21712,0 +21713,0 +21714,0 +21715,0 +21716,0 +21717,0 +21718,0 +21719,0 +21720,0 +21721,0 +21722,0 +21723,0 +21724,0 +21725,0 +21726,0 +21727,0 +21728,0 +21729,0 +21730,0 +21731,0 +21732,0 +21733,0 +21734,0 +21735,0 +21736,0 +21737,0 +21738,0 +21739,0 +21740,0 +21741,0 +21742,0 +21743,0 +21744,0 +21745,0 +21746,0 +21747,0 +21748,0 +21749,0 +21750,0 +21751,0 +21752,0 +21753,0 +21754,0 +21755,0 +21756,0 +21757,0 +21758,0 +21759,0 +21760,0 +21761,0 +21762,0 +21763,0 +21764,0 +21765,0 +21766,0 +21767,0 +21768,0 +21769,0 +21770,0 +21771,0 +21772,0 +21773,0 +21774,0 +21775,0 +21776,0 +21777,0 +21778,0 +21779,0 +21780,0 +21781,0 +21782,0 +21783,0 +21784,0 +21785,0 +21786,0 +21787,0 +21788,0 +21789,0 +21790,0 +21791,0 +21792,0 +21793,0 +21794,0 +21795,0 +21796,0 +21797,0 +21798,0 +21799,0 +21800,0 +21801,0 +21802,0 +21803,0 +21804,0 +21805,0 +21806,0 +21807,0 +21808,0 +21809,0 +21810,0 +21811,0 +21812,0 +21813,0 +21814,0 +21815,0 +21816,0 +21817,0 +21818,0 +21819,0 +21820,0 +21821,0 +21822,0 +21823,0 +21824,0 +21825,0 +21826,0 +21827,0 +21828,0 +21829,0 +21830,0 +21831,0 +21832,0 +21833,0 +21834,0 +21835,0 +21836,0 +21837,0 +21838,0 +21839,0 +21840,0 +21841,0 +21842,0 +21843,0 +21844,0 +21845,0 +21846,0 +21847,0 +21848,0 +21849,0 +21850,0 +21851,0 +21852,0 +21853,0 +21854,0 +21855,0 +21856,0 +21857,0 +21858,0 +21859,0 +21860,0 +21861,0 +21862,0 +21863,0 +21864,0 +21865,0 +21866,0 +21867,0 +21868,0 +21869,0 +21870,0 +21871,0 +21872,0 +21873,0 +21874,0 +21875,0 +21876,0 +21877,0 +21878,0 +21879,0 +21880,0 +21881,0 +21882,0 +21883,0 +21884,0 +21885,0 +21886,0 +21887,0 +21888,0 +21889,0 +21890,0 +21891,0 +21892,0 +21893,0 +21894,0 +21895,0 +21896,0 +21897,0 +21898,0 +21899,0 +21900,0 +21901,0 +21902,0 +21903,0 +21904,0 +21905,0 +21906,0 +21907,0 +21908,0 +21909,0 +21910,0 +21911,0 +21912,0 +21913,0 +21914,0 +21915,0 +21916,0 +21917,0 +21918,0 +21919,0 +21920,0 +21921,0 +21922,0 +21923,0 +21924,0 +21925,0 +21926,0 +21927,0 +21928,0 +21929,0 +21930,0 +21931,0 +21932,0 +21933,0 +21934,0 +21935,0 +21936,0 +21937,0 +21938,0 +21939,0 +21940,0 +21941,0 +21942,0 +21943,0 +21944,0 +21945,0 +21946,0 +21947,0 +21948,0 +21949,0 +21950,0 +21951,0 +21952,0 +21953,0 +21954,0 +21955,0 +21956,0 +21957,0 +21958,0 +21959,0 +21960,0 +21961,0 +21962,0 +21963,0 +21964,0 +21965,0 +21966,0 +21967,0 +21968,0 +21969,0 +21970,0 +21971,0 +21972,0 +21973,0 +21974,0 +21975,0 +21976,0 +21977,0 +21978,0 +21979,0 +21980,0 +21981,0 +21982,0 +21983,0 +21984,0 +21985,0 +21986,0 +21987,0 +21988,0 +21989,0 +21990,0 +21991,0 +21992,0 +21993,0 +21994,0 +21995,0 +21996,0 +21997,0 +21998,0 +21999,0 +22000,0 +22001,0 +22002,0 +22003,0 +22004,0 +22005,0 +22006,0 +22007,0 +22008,0 +22009,0 +22010,0 +22011,0 +22012,0 +22013,0 +22014,0 +22015,0 +22016,0 +22017,0 +22018,0 +22019,0 +22020,0 +22021,0 +22022,0 +22023,0 +22024,0 +22025,0 +22026,0 +22027,0 +22028,0 +22029,0 +22030,0 +22031,0 +22032,0 +22033,0 +22034,0 +22035,0 +22036,0 +22037,0 +22038,0 +22039,0 +22040,0 +22041,0 +22042,0 +22043,0 +22044,0 +22045,0 +22046,0 +22047,0 +22048,0 +22049,0 +22050,0 +22051,0 +22052,0 +22053,0 +22054,0 +22055,0 +22056,0 +22057,0 +22058,0 +22059,0 +22060,0 +22061,0 +22062,0 +22063,0 +22064,0 +22065,0 +22066,0 +22067,0 +22068,0 +22069,0 +22070,0 +22071,0 +22072,0 +22073,0 +22074,0 +22075,0 +22076,0 +22077,0 +22078,0 +22079,0 +22080,0 +22081,0 +22082,0 +22083,0 +22084,0 +22085,0 +22086,0 +22087,0 +22088,0 +22089,0 +22090,0 +22091,0 +22092,0 +22093,0 +22094,0 +22095,0 +22096,0 +22097,0 +22098,0 +22099,0 +22100,0 +22101,0 +22102,0 +22103,0 +22104,0 +22105,0 +22106,0 +22107,0 +22108,0 +22109,0 +22110,0 +22111,0 +22112,0 +22113,0 +22114,0 +22115,0 +22116,0 +22117,0 +22118,0 +22119,0 +22120,0 +22121,0 +22122,0 +22123,0 +22124,0 +22125,0 +22126,0 +22127,0 +22128,0 +22129,0 +22130,0 +22131,0 +22132,0 +22133,0 +22134,0 +22135,0 +22136,0 +22137,0 +22138,0 +22139,0 +22140,0 +22141,0 +22142,0 +22143,0 +22144,0 +22145,0 +22146,0 +22147,0 +22148,0 +22149,0 +22150,0 +22151,0 +22152,0 +22153,0 +22154,0 +22155,0 +22156,0 +22157,0 +22158,0 +22159,0 +22160,0 +22161,0 +22162,0 +22163,0 +22164,0 +22165,0 +22166,0 +22167,0 +22168,0 +22169,0 +22170,0 +22171,0 +22172,0 +22173,0 +22174,0 +22175,0 +22176,0 +22177,0 +22178,0 +22179,0 +22180,0 +22181,0 +22182,0 +22183,0 +22184,0 +22185,0 +22186,0 +22187,0 +22188,0 +22189,0 +22190,0 +22191,0 +22192,0 +22193,0 +22194,0 +22195,0 +22196,0 +22197,0 +22198,0 +22199,0 +22200,0 +22201,0 +22202,0 +22203,0 +22204,0 +22205,0 +22206,0 +22207,0 +22208,0 +22209,0 +22210,0 +22211,0 +22212,0 +22213,0 +22214,0 +22215,0 +22216,0 +22217,0 +22218,0 +22219,0 +22220,0 +22221,0 +22222,0 +22223,0 +22224,0 +22225,0 +22226,0 +22227,0 +22228,0 +22229,0 +22230,0 +22231,0 +22232,0 +22233,0 +22234,0 +22235,0 +22236,0 +22237,0 +22238,0 +22239,0 +22240,0 +22241,0 +22242,0 +22243,0 +22244,0 +22245,0 +22246,0 +22247,0 +22248,0 +22249,0 +22250,0 +22251,0 +22252,0 +22253,0 +22254,0 +22255,0 +22256,0 +22257,0 +22258,0 +22259,0 +22260,0 +22261,0 +22262,0 +22263,0 +22264,0 +22265,0 +22266,0 +22267,0 +22268,0 +22269,0 +22270,0 +22271,0 +22272,0 +22273,0 +22274,0 +22275,0 +22276,0 +22277,0 +22278,0 +22279,0 +22280,0 +22281,0 +22282,0 +22283,0 +22284,0 +22285,0 +22286,0 +22287,0 +22288,0 +22289,0 +22290,0 +22291,0 +22292,0 +22293,0 +22294,0 +22295,0 +22296,0 +22297,0 +22298,0 +22299,0 +22300,0 +22301,0 +22302,0 +22303,0 +22304,0 +22305,0 +22306,0 +22307,0 +22308,0 +22309,0 +22310,0 +22311,0 +22312,0 +22313,0 +22314,0 +22315,0 +22316,0 +22317,0 +22318,0 +22319,0 +22320,0 +22321,0 +22322,0 +22323,0 +22324,0 +22325,0 +22326,0 +22327,0 +22328,0 +22329,0 +22330,0 +22331,0 +22332,0 +22333,0 +22334,0 +22335,0 +22336,0 +22337,0 +22338,0 +22339,0 +22340,0 +22341,0 +22342,0 +22343,0 +22344,0 +22345,0 +22346,0 +22347,0 +22348,0 +22349,0 +22350,0 +22351,0 +22352,0 +22353,0 +22354,0 +22355,0 +22356,0 +22357,0 +22358,0 +22359,0 +22360,0 +22361,0 +22362,0 +22363,0 +22364,0 +22365,0 +22366,0 +22367,0 +22368,0 +22369,0 +22370,0 +22371,0 +22372,0 +22373,0 +22374,0 +22375,0 +22376,0 +22377,0 +22378,0 +22379,0 +22380,0 +22381,0 +22382,0 +22383,0 +22384,0 +22385,0 +22386,0 +22387,0 +22388,0 +22389,0 +22390,0 +22391,0 +22392,0 +22393,0 +22394,0 +22395,0 +22396,0 +22397,0 +22398,0 +22399,0 +22400,0 +22401,0 +22402,0 +22403,0 +22404,0 +22405,0 +22406,0 +22407,0 +22408,0 +22409,0 +22410,0 +22411,0 +22412,0 +22413,0 +22414,0 +22415,0 +22416,0 +22417,0 +22418,0 +22419,0 +22420,0 +22421,0 +22422,0 +22423,0 +22424,0 +22425,0 +22426,0 +22427,0 +22428,0 +22429,0 +22430,0 +22431,0 +22432,0 +22433,0 +22434,0 +22435,0 +22436,0 +22437,0 +22438,0 +22439,0 +22440,0 +22441,0 +22442,0 +22443,0 +22444,0 +22445,0 +22446,0 +22447,0 +22448,0 +22449,0 +22450,0 +22451,0 +22452,0 +22453,0 +22454,0 +22455,0 +22456,0 +22457,0 +22458,0 +22459,0 +22460,0 +22461,0 +22462,0 +22463,0 +22464,0 +22465,0 +22466,0 +22467,0 +22468,0 +22469,0 +22470,0 +22471,0 +22472,0 +22473,0 +22474,0 +22475,0 +22476,0 +22477,0 +22478,0 +22479,0 +22480,0 +22481,0 +22482,0 +22483,0 +22484,0 +22485,0 +22486,0 +22487,0 +22488,0 +22489,0 +22490,0 +22491,0 +22492,0 +22493,0 +22494,0 +22495,0 +22496,0 +22497,0 +22498,0 +22499,0 +22500,0 +22501,0 +22502,0 +22503,0 +22504,0 +22505,0 +22506,0 +22507,0 +22508,0 +22509,0 +22510,0 +22511,0 +22512,0 +22513,0 +22514,0 +22515,0 +22516,0 +22517,0 +22518,0 +22519,0 +22520,0 +22521,0 +22522,0 +22523,0 +22524,0 +22525,0 +22526,0 +22527,0 +22528,0 +22529,0 +22530,0 +22531,0 +22532,0 +22533,0 +22534,0 +22535,0 +22536,0 +22537,0 +22538,0 +22539,0 +22540,0 +22541,0 +22542,0 +22543,0 +22544,0 +22545,0 +22546,0 +22547,0 +22548,0 +22549,0 +22550,0 +22551,0 +22552,0 +22553,0 +22554,0 +22555,0 +22556,0 +22557,0 +22558,0 +22559,0 +22560,0 +22561,0 +22562,0 +22563,0 +22564,0 +22565,0 +22566,0 +22567,0 +22568,0 +22569,0 +22570,0 +22571,0 +22572,0 +22573,0 +22574,0 +22575,0 +22576,0 +22577,0 +22578,0 +22579,0 +22580,0 +22581,0 +22582,0 +22583,0 +22584,0 +22585,0 +22586,0 +22587,0 +22588,0 +22589,0 +22590,0 +22591,0 +22592,0 +22593,0 +22594,0 +22595,0 +22596,0 +22597,0 +22598,0 +22599,0 +22600,0 +22601,0 +22602,0 +22603,0 +22604,0 +22605,0 +22606,0 +22607,0 +22608,0 +22609,0 +22610,0 +22611,0 +22612,0 +22613,0 +22614,0 +22615,0 +22616,0 +22617,0 +22618,0 +22619,0 +22620,0 +22621,0 +22622,0 +22623,0 +22624,0 +22625,0 +22626,0 +22627,0 +22628,0 +22629,0 +22630,0 +22631,0 +22632,0 +22633,0 +22634,0 +22635,0 +22636,0 +22637,0 +22638,0 +22639,0 +22640,0 +22641,0 +22642,0 +22643,0 +22644,0 +22645,0 +22646,0 +22647,0 +22648,0 +22649,0 +22650,0 +22651,0 +22652,0 +22653,0 +22654,0 +22655,0 +22656,0 +22657,0 +22658,0 +22659,0 +22660,0 +22661,0 +22662,0 +22663,0 +22664,0 +22665,0 +22666,0 +22667,0 +22668,0 +22669,0 +22670,0 +22671,0 +22672,0 +22673,0 +22674,0 +22675,0 +22676,0 +22677,0 +22678,0 +22679,0 +22680,0 +22681,0 +22682,0 +22683,0 +22684,0 +22685,0 +22686,0 +22687,0 +22688,0 +22689,0 +22690,0 +22691,0 +22692,0 +22693,0 +22694,0 +22695,0 +22696,0 +22697,0 +22698,0 +22699,0 +22700,0 +22701,0 +22702,0 +22703,0 +22704,0 +22705,0 +22706,0 +22707,0 +22708,0 +22709,0 +22710,0 +22711,0 +22712,0 +22713,0 +22714,0 +22715,0 +22716,0 +22717,0 +22718,0 +22719,0 +22720,0 +22721,0 +22722,0 +22723,0 +22724,0 +22725,0 +22726,0 +22727,0 +22728,0 +22729,0 +22730,0 +22731,0 +22732,0 +22733,0 +22734,0 +22735,0 +22736,0 +22737,0 +22738,0 +22739,0 +22740,0 +22741,0 +22742,0 +22743,0 +22744,0 +22745,0 +22746,0 +22747,0 +22748,0 +22749,0 +22750,0 +22751,0 +22752,0 +22753,0 +22754,0 +22755,0 +22756,0 +22757,0 +22758,0 +22759,0 +22760,0 +22761,0 +22762,0 +22763,0 +22764,0 +22765,0 +22766,0 +22767,0 +22768,0 +22769,0 +22770,0 +22771,0 +22772,0 +22773,0 +22774,0 +22775,0 +22776,0 +22777,0 +22778,0 +22779,0 +22780,0 +22781,0 +22782,0 +22783,0 +22784,0 +22785,0 +22786,0 +22787,0 +22788,0 +22789,0 +22790,0 +22791,0 +22792,0 +22793,0 +22794,0 +22795,0 +22796,0 +22797,0 +22798,0 +22799,0 +22800,0 +22801,0 +22802,0 +22803,0 +22804,0 +22805,0 +22806,0 +22807,0 +22808,0 +22809,0 +22810,0 +22811,0 +22812,0 +22813,0 +22814,0 +22815,0 +22816,0 +22817,0 +22818,0 +22819,0 +22820,0 +22821,0 +22822,0 +22823,0 +22824,0 +22825,0 +22826,0 +22827,0 +22828,0 +22829,0 +22830,0 +22831,0 +22832,0 +22833,0 +22834,0 +22835,0 +22836,0 +22837,0 +22838,0 +22839,0 +22840,0 +22841,0 +22842,0 +22843,0 +22844,0 +22845,0 +22846,0 +22847,0 +22848,0 +22849,0 +22850,0 +22851,0 +22852,0 +22853,0 +22854,0 +22855,0 +22856,0 +22857,0 +22858,0 +22859,0 +22860,0 +22861,0 +22862,0 +22863,0 +22864,0 +22865,0 +22866,0 +22867,0 +22868,0 +22869,0 +22870,0 +22871,0 +22872,0 +22873,0 +22874,0 +22875,0 +22876,0 +22877,0 +22878,0 +22879,0 +22880,0 +22881,0 +22882,0 +22883,0 +22884,0 +22885,0 +22886,0 +22887,0 +22888,0 +22889,0 +22890,0 +22891,0 +22892,0 +22893,0 +22894,0 +22895,0 +22896,0 +22897,0 +22898,0 +22899,0 +22900,0 +22901,0 +22902,0 +22903,0 +22904,0 +22905,0 +22906,0 +22907,0 +22908,0 +22909,0 +22910,0 +22911,0 +22912,0 +22913,0 +22914,0 +22915,0 +22916,0 +22917,0 +22918,0 +22919,0 +22920,0 +22921,0 +22922,0 +22923,0 +22924,0 +22925,0 +22926,0 +22927,0 +22928,0 +22929,0 +22930,0 +22931,0 +22932,0 +22933,0 +22934,0 +22935,0 +22936,0 +22937,0 +22938,0 +22939,0 +22940,0 +22941,0 +22942,0 +22943,0 +22944,0 +22945,0 +22946,0 +22947,0 +22948,0 +22949,0 +22950,0 +22951,0 +22952,0 +22953,0 +22954,0 +22955,0 +22956,0 +22957,0 +22958,0 +22959,0 +22960,0 +22961,0 +22962,0 +22963,0 +22964,0 +22965,0 +22966,0 +22967,0 +22968,0 +22969,0 +22970,0 +22971,0 +22972,0 +22973,0 +22974,0 +22975,0 +22976,0 +22977,0 +22978,0 +22979,0 +22980,0 +22981,0 +22982,0 +22983,0 +22984,0 +22985,0 +22986,0 +22987,0 +22988,0 +22989,0 +22990,0 +22991,0 +22992,0 +22993,0 +22994,0 +22995,0 +22996,0 +22997,0 +22998,0 +22999,0 +23000,0 +23001,0 +23002,0 +23003,0 +23004,0 +23005,0 +23006,0 +23007,0 +23008,0 +23009,0 +23010,0 +23011,0 +23012,0 +23013,0 +23014,0 +23015,0 +23016,0 +23017,0 +23018,0 +23019,0 +23020,0 +23021,0 +23022,0 +23023,0 +23024,0 +23025,0 +23026,0 +23027,0 +23028,0 +23029,0 +23030,0 +23031,0 +23032,0 +23033,0 +23034,0 +23035,0 +23036,0 +23037,0 +23038,0 +23039,0 +23040,0 +23041,0 +23042,0 +23043,0 +23044,0 +23045,0 +23046,0 +23047,0 +23048,0 +23049,0 +23050,0 +23051,0 +23052,0 +23053,0 +23054,0 +23055,0 +23056,0 +23057,0 +23058,0 +23059,0 +23060,0 +23061,0 +23062,0 +23063,0 +23064,0 +23065,0 +23066,0 +23067,0 +23068,0 +23069,0 +23070,0 +23071,0 +23072,0 +23073,0 +23074,0 +23075,0 +23076,0 +23077,0 +23078,0 +23079,0 +23080,0 +23081,0 +23082,0 +23083,0 +23084,0 +23085,0 +23086,0 +23087,0 +23088,0 +23089,0 +23090,0 +23091,0 +23092,0 +23093,0 +23094,0 +23095,0 +23096,0 +23097,0 +23098,0 +23099,0 +23100,0 +23101,0 +23102,0 +23103,0 +23104,0 +23105,0 +23106,0 +23107,0 +23108,0 +23109,0 +23110,0 +23111,0 +23112,0 +23113,0 +23114,0 +23115,0 +23116,0 +23117,0 +23118,0 +23119,0 +23120,0 +23121,0 +23122,0 +23123,0 +23124,0 +23125,0 +23126,0 +23127,0 +23128,0 +23129,0 +23130,0 +23131,0 +23132,0 +23133,0 +23134,0 +23135,0 +23136,0 +23137,0 +23138,0 +23139,0 +23140,0 +23141,0 +23142,0 +23143,0 +23144,0 +23145,0 +23146,0 +23147,0 +23148,0 +23149,0 +23150,0 +23151,0 +23152,0 +23153,0 +23154,0 +23155,0 +23156,0 +23157,0 +23158,0 +23159,0 +23160,0 +23161,0 +23162,0 +23163,0 +23164,0 +23165,0 +23166,0 +23167,0 +23168,0 +23169,0 +23170,0 +23171,0 +23172,0 +23173,0 +23174,0 +23175,0 +23176,0 +23177,0 +23178,0 +23179,0 +23180,0 +23181,0 +23182,0 +23183,0 +23184,0 +23185,0 +23186,0 +23187,0 +23188,0 +23189,0 +23190,0 +23191,0 +23192,0 +23193,0 +23194,0 +23195,0 +23196,0 +23197,0 +23198,0 +23199,0 +23200,0 +23201,0 +23202,0 +23203,0 +23204,0 +23205,0 +23206,0 +23207,0 +23208,0 +23209,0 +23210,0 +23211,0 +23212,0 +23213,0 +23214,0 +23215,0 +23216,0 +23217,0 +23218,0 +23219,0 +23220,0 +23221,0 +23222,0 +23223,0 +23224,0 +23225,0 +23226,0 +23227,0 +23228,0 +23229,0 +23230,0 +23231,0 +23232,0 +23233,0 +23234,0 +23235,0 +23236,0 +23237,0 +23238,0 +23239,0 +23240,0 +23241,0 +23242,0 +23243,0 +23244,0 +23245,0 +23246,0 +23247,0 +23248,0 +23249,0 +23250,0 +23251,0 +23252,0 +23253,0 +23254,0 +23255,0 +23256,0 +23257,0 +23258,0 +23259,0 +23260,0 +23261,0 +23262,0 +23263,0 +23264,0 +23265,0 +23266,0 +23267,0 +23268,0 +23269,0 +23270,0 +23271,0 +23272,0 +23273,0 +23274,0 +23275,0 +23276,0 +23277,0 +23278,0 +23279,0 +23280,0 +23281,0 +23282,0 +23283,0 +23284,0 +23285,0 +23286,0 +23287,0 +23288,0 +23289,0 +23290,0 +23291,0 +23292,0 +23293,0 +23294,0 +23295,0 +23296,0 +23297,0 +23298,0 +23299,0 +23300,0 +23301,0 +23302,0 +23303,0 +23304,0 +23305,0 +23306,0 +23307,0 +23308,0 +23309,0 +23310,0 +23311,0 +23312,0 +23313,0 +23314,0 +23315,0 +23316,0 +23317,0 +23318,0 +23319,0 +23320,0 +23321,0 +23322,0 +23323,0 +23324,0 +23325,0 +23326,0 +23327,0 +23328,0 +23329,0 +23330,0 +23331,0 +23332,0 +23333,0 +23334,0 +23335,0 +23336,0 +23337,0 +23338,0 +23339,0 +23340,0 +23341,0 +23342,0 +23343,0 +23344,0 +23345,0 +23346,0 +23347,0 +23348,0 +23349,0 +23350,0 +23351,0 +23352,0 +23353,0 +23354,0 +23355,0 +23356,0 +23357,0 +23358,0 +23359,0 +23360,0 +23361,0 +23362,0 +23363,0 +23364,0 +23365,0 +23366,0 +23367,0 +23368,0 +23369,0 +23370,0 +23371,0 +23372,0 +23373,0 +23374,0 +23375,0 +23376,0 +23377,0 +23378,0 +23379,0 +23380,0 +23381,0 +23382,0 +23383,0 +23384,0 +23385,0 +23386,0 +23387,0 +23388,0 +23389,0 +23390,0 +23391,0 +23392,0 +23393,0 +23394,0 +23395,0 +23396,0 +23397,0 +23398,0 +23399,0 +23400,0 +23401,0 +23402,0 +23403,0 +23404,0 +23405,0 +23406,0 +23407,0 +23408,0 +23409,0 +23410,0 +23411,0 +23412,0 +23413,0 +23414,0 +23415,0 +23416,0 +23417,0 +23418,0 +23419,0 +23420,0 +23421,0 +23422,0 +23423,0 +23424,0 +23425,0 +23426,0 +23427,0 +23428,0 +23429,0 +23430,0 +23431,0 +23432,0 +23433,0 +23434,0 +23435,0 +23436,0 +23437,0 +23438,0 +23439,0 +23440,0 +23441,0 +23442,0 +23443,0 +23444,0 +23445,0 +23446,0 +23447,0 +23448,0 +23449,0 +23450,0 +23451,0 +23452,0 +23453,0 +23454,0 +23455,0 +23456,0 +23457,0 +23458,0 +23459,0 +23460,0 +23461,0 +23462,0 +23463,0 +23464,0 +23465,0 +23466,0 +23467,0 +23468,0 +23469,0 +23470,0 +23471,0 +23472,0 +23473,0 +23474,0 +23475,0 +23476,0 +23477,0 +23478,0 +23479,0 +23480,0 +23481,0 +23482,0 +23483,0 +23484,0 +23485,0 +23486,0 +23487,0 +23488,0 +23489,0 +23490,0 +23491,0 +23492,0 +23493,0 +23494,0 +23495,0 +23496,0 +23497,0 +23498,0 +23499,0 +23500,0 +23501,0 +23502,0 +23503,0 +23504,0 +23505,0 +23506,0 +23507,0 +23508,0 +23509,0 +23510,0 +23511,0 +23512,0 +23513,0 +23514,0 +23515,0 +23516,0 +23517,0 +23518,0 +23519,0 +23520,0 +23521,0 +23522,0 +23523,0 +23524,0 +23525,0 +23526,0 +23527,0 +23528,0 +23529,0 +23530,0 +23531,0 +23532,0 +23533,0 +23534,0 +23535,0 +23536,0 +23537,0 +23538,0 +23539,0 +23540,0 +23541,0 +23542,0 +23543,0 +23544,0 +23545,0 +23546,0 +23547,0 +23548,0 +23549,0 +23550,0 +23551,0 +23552,0 +23553,0 +23554,0 +23555,0 +23556,0 +23557,0 +23558,0 +23559,0 +23560,0 +23561,0 +23562,0 +23563,0 +23564,0 +23565,0 +23566,0 +23567,0 +23568,0 +23569,0 +23570,0 +23571,0 +23572,0 +23573,0 +23574,0 +23575,0 +23576,0 +23577,0 +23578,0 +23579,0 +23580,0 +23581,0 +23582,0 +23583,0 +23584,0 +23585,0 +23586,0 +23587,0 +23588,0 +23589,0 +23590,0 +23591,0 +23592,0 +23593,0 +23594,0 +23595,0 +23596,0 +23597,0 +23598,0 +23599,0 +23600,0 +23601,0 +23602,0 +23603,0 +23604,0 +23605,0 +23606,0 +23607,0 +23608,0 +23609,0 +23610,0 +23611,0 +23612,0 +23613,0 +23614,0 +23615,0 +23616,0 +23617,0 +23618,0 +23619,0 +23620,0 +23621,0 +23622,0 +23623,0 +23624,0 +23625,0 +23626,0 +23627,0 +23628,0 +23629,0 +23630,0 +23631,0 +23632,0 +23633,0 +23634,0 +23635,0 +23636,0 +23637,0 +23638,0 +23639,0 +23640,0 +23641,0 +23642,0 +23643,0 +23644,0 +23645,0 +23646,0 +23647,0 +23648,0 +23649,0 +23650,0 +23651,0 +23652,0 +23653,0 +23654,0 +23655,0 +23656,0 +23657,0 +23658,0 +23659,0 +23660,0 +23661,0 +23662,0 +23663,0 +23664,0 +23665,0 +23666,0 +23667,0 +23668,0 +23669,0 +23670,0 +23671,0 +23672,0 +23673,0 +23674,0 +23675,0 +23676,0 +23677,0 +23678,0 +23679,0 +23680,0 +23681,0 +23682,0 +23683,0 +23684,0 +23685,0 +23686,0 +23687,0 +23688,0 +23689,0 +23690,0 +23691,0 +23692,0 +23693,0 +23694,0 +23695,0 +23696,0 +23697,0 +23698,0 +23699,0 +23700,0 +23701,0 +23702,0 +23703,0 +23704,0 +23705,0 +23706,0 +23707,0 +23708,0 +23709,0 +23710,0 +23711,0 +23712,0 +23713,0 +23714,0 +23715,0 +23716,0 +23717,0 +23718,0 +23719,0 +23720,0 +23721,0 +23722,0 +23723,0 +23724,0 +23725,0 +23726,0 +23727,0 +23728,0 +23729,0 +23730,0 +23731,0 +23732,0 +23733,0 +23734,0 +23735,0 +23736,0 +23737,0 +23738,0 +23739,0 +23740,0 +23741,0 +23742,0 +23743,0 +23744,0 +23745,0 +23746,0 +23747,0 +23748,0 +23749,0 +23750,0 +23751,0 +23752,0 +23753,0 +23754,0 +23755,0 +23756,0 +23757,0 +23758,0 +23759,0 +23760,0 +23761,0 +23762,0 +23763,0 +23764,0 +23765,0 +23766,0 +23767,0 +23768,0 +23769,0 +23770,0 +23771,0 +23772,0 +23773,0 +23774,0 +23775,0 +23776,0 +23777,0 +23778,0 +23779,0 +23780,0 +23781,0 +23782,0 +23783,0 +23784,0 +23785,0 +23786,0 +23787,0 +23788,0 +23789,0 +23790,0 +23791,0 +23792,0 +23793,0 +23794,0 +23795,0 +23796,0 +23797,0 +23798,0 +23799,0 +23800,0 +23801,0 +23802,0 +23803,0 +23804,0 +23805,0 +23806,0 +23807,0 +23808,0 +23809,0 +23810,0 +23811,0 +23812,0 +23813,0 +23814,0 +23815,0 +23816,0 +23817,0 +23818,0 +23819,0 +23820,0 +23821,0 +23822,0 +23823,0 +23824,0 +23825,0 +23826,0 +23827,0 +23828,0 +23829,0 +23830,0 +23831,0 +23832,0 +23833,0 +23834,0 +23835,0 +23836,0 +23837,0 +23838,0 +23839,0 +23840,0 +23841,0 +23842,0 +23843,0 +23844,0 +23845,0 +23846,0 +23847,0 +23848,0 +23849,0 +23850,0 +23851,0 +23852,0 +23853,0 +23854,0 +23855,0 +23856,0 +23857,0 +23858,0 +23859,0 +23860,0 +23861,0 +23862,0 +23863,0 +23864,0 +23865,0 +23866,0 +23867,0 +23868,0 +23869,0 +23870,0 +23871,0 +23872,0 +23873,0 +23874,0 +23875,0 +23876,0 +23877,0 +23878,0 +23879,0 +23880,0 +23881,0 +23882,0 +23883,0 +23884,0 +23885,0 +23886,0 +23887,0 +23888,0 +23889,0 +23890,0 +23891,0 +23892,0 +23893,0 +23894,0 +23895,0 +23896,0 +23897,0 +23898,0 +23899,0 +23900,0 +23901,0 +23902,0 +23903,0 +23904,0 +23905,0 +23906,0 +23907,0 +23908,0 +23909,0 +23910,0 +23911,0 +23912,0 +23913,0 +23914,0 +23915,0 +23916,0 +23917,0 +23918,0 +23919,0 +23920,0 +23921,0 +23922,0 +23923,0 +23924,0 +23925,0 +23926,0 +23927,0 +23928,0 +23929,0 +23930,0 +23931,0 +23932,0 +23933,0 +23934,0 +23935,0 +23936,0 +23937,0 +23938,0 +23939,0 +23940,0 +23941,0 +23942,0 +23943,0 +23944,0 +23945,0 +23946,0 +23947,0 +23948,0 +23949,0 +23950,0 +23951,0 +23952,0 +23953,0 +23954,0 +23955,0 +23956,0 +23957,0 +23958,0 +23959,0 +23960,0 +23961,0 +23962,0 +23963,0 +23964,0 +23965,0 +23966,0 +23967,0 +23968,0 +23969,0 +23970,0 +23971,0 +23972,0 +23973,0 +23974,0 +23975,0 +23976,0 +23977,0 +23978,0 +23979,0 +23980,0 +23981,0 +23982,0 +23983,0 +23984,0 +23985,0 +23986,0 +23987,0 +23988,0 +23989,0 +23990,0 +23991,0 +23992,0 +23993,0 +23994,0 +23995,0 +23996,0 +23997,0 +23998,0 +23999,0 +24000,0 +24001,0 +24002,0 +24003,0 +24004,0 +24005,0 +24006,0 +24007,0 +24008,0 +24009,0 +24010,0 +24011,0 +24012,0 +24013,0 +24014,0 +24015,0 +24016,0 +24017,0 +24018,0 +24019,0 +24020,0 +24021,0 +24022,0 +24023,0 +24024,0 +24025,0 +24026,0 +24027,0 +24028,0 +24029,0 +24030,0 +24031,0 +24032,0 +24033,0 +24034,0 +24035,0 +24036,0 +24037,0 +24038,0 +24039,0 +24040,0 +24041,0 +24042,0 +24043,0 +24044,0 +24045,0 +24046,0 +24047,0 +24048,0 +24049,0 +24050,0 +24051,0 +24052,0 +24053,0 +24054,0 +24055,0 +24056,0 +24057,0 +24058,0 +24059,0 +24060,0 +24061,0 +24062,0 +24063,0 +24064,0 +24065,0 +24066,0 +24067,0 +24068,0 +24069,0 +24070,0 +24071,0 +24072,0 +24073,0 +24074,0 +24075,0 +24076,0 +24077,0 +24078,0 +24079,0 +24080,0 +24081,0 +24082,0 +24083,0 +24084,0 +24085,0 +24086,0 +24087,0 +24088,0 +24089,0 +24090,0 +24091,0 +24092,0 +24093,0 +24094,0 +24095,0 +24096,0 +24097,0 +24098,0 +24099,0 +24100,0 +24101,0 +24102,0 +24103,0 +24104,0 +24105,0 +24106,0 +24107,0 +24108,0 +24109,0 +24110,0 +24111,0 +24112,0 +24113,0 +24114,0 +24115,0 +24116,0 +24117,0 +24118,0 +24119,0 +24120,0 +24121,0 +24122,0 +24123,0 +24124,0 +24125,0 +24126,0 +24127,0 +24128,0 +24129,0 +24130,0 +24131,0 +24132,0 +24133,0 +24134,0 +24135,0 +24136,0 +24137,0 +24138,0 +24139,0 +24140,0 +24141,0 +24142,0 +24143,0 +24144,0 +24145,0 +24146,0 +24147,0 +24148,0 +24149,0 +24150,0 +24151,0 +24152,0 +24153,0 +24154,0 +24155,0 +24156,0 +24157,0 +24158,0 +24159,0 +24160,0 +24161,0 +24162,0 +24163,0 +24164,0 +24165,0 +24166,0 +24167,0 +24168,0 +24169,0 +24170,0 +24171,0 +24172,0 +24173,0 +24174,0 +24175,0 +24176,0 +24177,0 +24178,0 +24179,0 +24180,0 +24181,0 +24182,0 +24183,0 +24184,0 +24185,0 +24186,0 +24187,0 +24188,0 +24189,0 +24190,0 +24191,0 +24192,0 +24193,0 +24194,0 +24195,0 +24196,0 +24197,0 +24198,0 +24199,0 +24200,0 +24201,0 +24202,0 +24203,0 +24204,0 +24205,0 +24206,0 +24207,0 +24208,0 +24209,0 +24210,0 +24211,0 +24212,0 +24213,0 +24214,0 +24215,0 +24216,0 +24217,0 +24218,0 +24219,0 +24220,0 +24221,0 +24222,0 +24223,0 +24224,0 +24225,0 +24226,0 +24227,0 +24228,0 +24229,0 +24230,0 +24231,0 +24232,0 +24233,0 +24234,0 +24235,0 +24236,0 +24237,0 +24238,0 +24239,0 +24240,0 +24241,0 +24242,0 +24243,0 +24244,0 +24245,0 +24246,0 +24247,0 +24248,0 +24249,0 +24250,0 +24251,0 +24252,0 +24253,0 +24254,0 +24255,0 +24256,0 +24257,0 +24258,0 +24259,0 +24260,0 +24261,0 +24262,0 +24263,0 +24264,0 +24265,0 +24266,0 +24267,0 +24268,0 +24269,0 +24270,0 +24271,0 +24272,0 +24273,0 +24274,0 +24275,0 +24276,0 +24277,0 +24278,0 +24279,0 +24280,0 +24281,0 +24282,0 +24283,0 +24284,0 +24285,0 +24286,0 +24287,0 +24288,0 +24289,0 +24290,0 +24291,0 +24292,0 +24293,0 +24294,0 +24295,0 +24296,0 +24297,0 +24298,0 +24299,0 +24300,0 +24301,0 +24302,0 +24303,0 +24304,0 +24305,0 +24306,0 +24307,0 +24308,0 +24309,0 +24310,0 +24311,0 +24312,0 +24313,0 +24314,0 +24315,0 +24316,0 +24317,0 +24318,0 +24319,0 +24320,0 +24321,0 +24322,0 +24323,0 +24324,0 +24325,0 +24326,0 +24327,0 +24328,0 +24329,0 +24330,0 +24331,0 +24332,0 +24333,0 +24334,0 +24335,0 +24336,0 +24337,0 +24338,0 +24339,0 +24340,0 +24341,0 +24342,0 +24343,0 +24344,0 +24345,0 +24346,0 +24347,0 +24348,0 +24349,0 +24350,0 +24351,0 +24352,0 +24353,0 +24354,0 +24355,0 +24356,0 +24357,0 +24358,0 +24359,0 +24360,0 +24361,0 +24362,0 +24363,0 +24364,0 +24365,0 +24366,0 +24367,0 +24368,0 +24369,0 +24370,0 +24371,0 +24372,0 +24373,0 +24374,0 +24375,0 +24376,0 +24377,0 +24378,0 +24379,0 +24380,0 +24381,0 +24382,0 +24383,0 +24384,0 +24385,0 +24386,0 +24387,0 +24388,0 +24389,0 +24390,0 +24391,0 +24392,0 +24393,0 +24394,0 +24395,0 +24396,0 +24397,0 +24398,0 +24399,0 +24400,0 +24401,0 +24402,0 +24403,0 +24404,0 +24405,0 +24406,0 +24407,0 +24408,0 +24409,0 +24410,0 +24411,0 +24412,0 +24413,0 +24414,0 +24415,0 +24416,0 +24417,0 +24418,0 +24419,0 +24420,0 +24421,0 +24422,0 +24423,0 +24424,0 +24425,0 +24426,0 +24427,0 +24428,0 +24429,0 +24430,0 +24431,0 +24432,0 +24433,0 +24434,0 +24435,0 +24436,0 +24437,0 +24438,0 +24439,0 +24440,0 +24441,0 +24442,0 +24443,0 +24444,0 +24445,0 +24446,0 +24447,0 +24448,0 +24449,0 +24450,0 +24451,0 +24452,0 +24453,0 +24454,0 +24455,0 +24456,0 +24457,0 +24458,0 +24459,0 +24460,0 +24461,0 +24462,0 +24463,0 +24464,0 +24465,0 +24466,0 +24467,0 +24468,0 +24469,0 +24470,0 +24471,0 +24472,0 +24473,0 +24474,0 +24475,0 +24476,0 +24477,0 +24478,0 +24479,0 +24480,0 +24481,0 +24482,0 +24483,0 +24484,0 +24485,0 +24486,0 +24487,0 +24488,0 +24489,0 +24490,0 +24491,0 +24492,0 +24493,0 +24494,0 +24495,0 +24496,0 +24497,0 +24498,0 +24499,0 +24500,0 +24501,0 +24502,0 +24503,0 +24504,0 +24505,0 +24506,0 +24507,0 +24508,0 +24509,0 +24510,0 +24511,0 +24512,0 +24513,0 +24514,0 +24515,0 +24516,0 +24517,0 +24518,0 +24519,0 +24520,0 +24521,0 +24522,0 +24523,0 +24524,0 +24525,0 +24526,0 +24527,0 +24528,0 +24529,0 +24530,0 +24531,0 +24532,0 +24533,0 +24534,0 +24535,0 +24536,0 +24537,0 +24538,0 +24539,0 +24540,0 +24541,0 +24542,0 +24543,0 +24544,0 +24545,0 +24546,0 +24547,0 +24548,0 +24549,0 +24550,0 +24551,0 +24552,0 +24553,0 +24554,0 +24555,0 +24556,0 +24557,0 +24558,0 +24559,0 +24560,0 +24561,0 +24562,0 +24563,0 +24564,0 +24565,0 +24566,0 +24567,0 +24568,0 +24569,0 +24570,0 +24571,0 +24572,0 +24573,0 +24574,0 +24575,0 +24576,0 +24577,0 +24578,0 +24579,0 +24580,0 +24581,0 +24582,0 +24583,0 +24584,0 +24585,0 +24586,0 +24587,0 +24588,0 +24589,0 +24590,0 +24591,0 +24592,0 +24593,0 +24594,0 +24595,0 +24596,0 +24597,0 +24598,0 +24599,0 +24600,0 +24601,0 +24602,0 +24603,0 +24604,0 +24605,0 +24606,0 +24607,0 +24608,0 +24609,0 +24610,0 +24611,0 +24612,0 +24613,0 +24614,0 +24615,0 +24616,0 +24617,0 +24618,0 +24619,0 +24620,0 +24621,0 +24622,0 +24623,0 +24624,0 +24625,0 +24626,0 +24627,0 +24628,0 +24629,0 +24630,0 +24631,0 +24632,0 +24633,0 +24634,0 +24635,0 +24636,0 +24637,0 +24638,0 +24639,0 +24640,0 +24641,0 +24642,0 +24643,0 +24644,0 +24645,0 +24646,0 +24647,0 +24648,0 +24649,0 +24650,0 +24651,0 +24652,0 +24653,0 +24654,0 +24655,0 +24656,0 +24657,0 +24658,0 +24659,0 +24660,0 +24661,0 +24662,0 +24663,0 +24664,0 +24665,0 +24666,0 +24667,0 +24668,0 +24669,0 +24670,0 +24671,0 +24672,0 +24673,0 +24674,0 +24675,0 +24676,0 +24677,0 +24678,0 +24679,0 +24680,0 +24681,0 +24682,0 +24683,0 +24684,0 +24685,0 +24686,0 +24687,0 +24688,0 +24689,0 +24690,0 +24691,0 +24692,0 +24693,0 +24694,0 +24695,0 +24696,0 +24697,0 +24698,0 +24699,0 +24700,0 +24701,0 +24702,0 +24703,0 +24704,0 +24705,0 +24706,0 +24707,0 +24708,0 +24709,0 +24710,0 +24711,0 +24712,0 +24713,0 +24714,0 +24715,0 +24716,0 +24717,0 +24718,0 +24719,0 +24720,0 +24721,0 +24722,0 +24723,0 +24724,0 +24725,0 +24726,0 +24727,0 +24728,0 +24729,0 +24730,0 +24731,0 +24732,0 +24733,0 +24734,0 +24735,0 +24736,0 +24737,0 +24738,0 +24739,0 +24740,0 +24741,0 +24742,0 +24743,0 +24744,0 +24745,0 +24746,0 +24747,0 +24748,0 +24749,0 +24750,0 +24751,0 +24752,0 +24753,0 +24754,0 +24755,0 +24756,0 +24757,0 +24758,0 +24759,0 +24760,0 +24761,0 +24762,0 +24763,0 +24764,0 +24765,0 +24766,0 +24767,0 +24768,0 +24769,0 +24770,0 +24771,0 +24772,0 +24773,0 +24774,0 +24775,0 +24776,0 +24777,0 +24778,0 +24779,0 +24780,0 +24781,0 +24782,0 +24783,0 +24784,0 +24785,0 +24786,0 +24787,0 +24788,0 +24789,0 +24790,0 +24791,0 +24792,0 +24793,0 +24794,0 +24795,0 +24796,0 +24797,0 +24798,0 +24799,0 +24800,0 +24801,0 +24802,0 +24803,0 +24804,0 +24805,0 +24806,0 +24807,0 +24808,0 +24809,0 +24810,0 +24811,0 +24812,0 +24813,0 +24814,0 +24815,0 +24816,0 +24817,0 +24818,0 +24819,0 +24820,0 +24821,0 +24822,0 +24823,0 +24824,0 +24825,0 +24826,0 +24827,0 +24828,0 +24829,0 +24830,0 +24831,0 +24832,0 +24833,0 +24834,0 +24835,0 +24836,0 +24837,0 +24838,0 +24839,0 +24840,0 +24841,0 +24842,0 +24843,0 +24844,0 +24845,0 +24846,0 +24847,0 +24848,0 +24849,0 +24850,0 +24851,0 +24852,0 +24853,0 +24854,0 +24855,0 +24856,0 +24857,0 +24858,0 +24859,0 +24860,0 +24861,0 +24862,0 +24863,0 +24864,0 +24865,0 +24866,0 +24867,0 +24868,0 +24869,0 +24870,0 +24871,0 +24872,0 +24873,0 +24874,0 +24875,0 +24876,0 +24877,0 +24878,0 +24879,0 +24880,0 +24881,0 +24882,0 +24883,0 +24884,0 +24885,0 +24886,0 +24887,0 +24888,0 +24889,0 +24890,0 +24891,0 +24892,0 +24893,0 +24894,0 +24895,0 +24896,0 +24897,0 +24898,0 +24899,0 +24900,0 +24901,0 +24902,0 +24903,0 +24904,0 +24905,0 +24906,0 +24907,0 +24908,0 +24909,0 +24910,0 +24911,0 +24912,0 +24913,0 +24914,0 +24915,0 +24916,0 +24917,0 +24918,0 +24919,0 +24920,0 +24921,0 +24922,0 +24923,0 +24924,0 +24925,0 +24926,0 +24927,0 +24928,0 +24929,0 +24930,0 +24931,0 +24932,0 +24933,0 +24934,0 +24935,0 +24936,0 +24937,0 +24938,0 +24939,0 +24940,0 +24941,0 +24942,0 +24943,0 +24944,0 +24945,0 +24946,0 +24947,0 +24948,0 +24949,0 +24950,0 +24951,0 +24952,0 +24953,0 +24954,0 +24955,0 +24956,0 +24957,0 +24958,0 +24959,0 +24960,0 +24961,0 +24962,0 +24963,0 +24964,0 +24965,0 +24966,0 +24967,0 +24968,0 +24969,0 +24970,0 +24971,0 +24972,0 +24973,0 +24974,0 +24975,0 +24976,0 +24977,0 +24978,0 +24979,0 +24980,0 +24981,0 +24982,0 +24983,0 +24984,0 +24985,0 +24986,0 +24987,0 +24988,0 +24989,0 +24990,0 +24991,0 +24992,0 +24993,0 +24994,0 +24995,0 +24996,0 +24997,0 +24998,0 +24999,0 +25000,0 +25001,0 +25002,0 +25003,0 +25004,0 +25005,0 +25006,0 +25007,0 +25008,0 +25009,0 +25010,0 +25011,0 +25012,0 +25013,0 +25014,0 +25015,0 +25016,0 +25017,0 +25018,0 +25019,0 +25020,0 +25021,0 +25022,0 +25023,0 +25024,0 +25025,0 +25026,0 +25027,0 +25028,0 +25029,0 +25030,0 +25031,0 +25032,0 +25033,0 +25034,0 +25035,0 +25036,0 +25037,0 +25038,0 +25039,0 +25040,0 +25041,0 +25042,0 +25043,0 +25044,0 +25045,0 +25046,0 +25047,0 +25048,0 +25049,0 +25050,0 +25051,0 +25052,0 +25053,0 +25054,0 +25055,0 +25056,0 +25057,0 +25058,0 +25059,0 +25060,0 +25061,0 +25062,0 +25063,0 +25064,0 +25065,0 +25066,0 +25067,0 +25068,0 +25069,0 +25070,0 +25071,0 +25072,0 +25073,0 +25074,0 +25075,0 +25076,0 +25077,0 +25078,0 +25079,0 +25080,0 +25081,0 +25082,0 +25083,0 +25084,0 +25085,0 +25086,0 +25087,0 +25088,0 +25089,0 +25090,0 +25091,0 +25092,0 +25093,0 +25094,0 +25095,0 +25096,0 +25097,0 +25098,0 +25099,0 +25100,0 +25101,0 +25102,0 +25103,0 +25104,0 +25105,0 +25106,0 +25107,0 +25108,0 +25109,0 +25110,0 +25111,0 +25112,0 +25113,0 +25114,0 +25115,0 +25116,0 +25117,0 +25118,0 +25119,0 +25120,0 +25121,0 +25122,0 +25123,0 +25124,0 +25125,0 +25126,0 +25127,0 +25128,0 +25129,0 +25130,0 +25131,0 +25132,0 +25133,0 +25134,0 +25135,0 +25136,0 +25137,0 +25138,0 +25139,0 +25140,0 +25141,0 +25142,0 +25143,0 +25144,0 +25145,0 +25146,0 +25147,0 +25148,0 +25149,0 +25150,0 +25151,0 +25152,0 +25153,0 +25154,0 +25155,0 +25156,0 +25157,0 +25158,0 +25159,0 +25160,0 +25161,0 +25162,0 +25163,0 +25164,0 +25165,0 +25166,0 +25167,0 +25168,0 +25169,0 +25170,0 +25171,0 +25172,0 +25173,0 +25174,0 +25175,0 +25176,0 +25177,0 +25178,0 +25179,0 +25180,0 +25181,0 +25182,0 +25183,0 +25184,0 +25185,0 +25186,0 +25187,0 +25188,0 +25189,0 +25190,0 +25191,0 +25192,0 +25193,0 +25194,0 +25195,0 +25196,0 +25197,0 +25198,0 +25199,0 +25200,0 +25201,0 +25202,0 +25203,0 +25204,0 +25205,0 +25206,0 +25207,0 +25208,0 +25209,0 +25210,0 +25211,0 +25212,0 +25213,0 +25214,0 +25215,0 +25216,0 +25217,0 +25218,0 +25219,0 +25220,0 +25221,0 +25222,0 +25223,0 +25224,0 +25225,0 +25226,0 +25227,0 +25228,0 +25229,0 +25230,0 +25231,0 +25232,0 +25233,0 +25234,0 +25235,0 +25236,0 +25237,0 +25238,0 +25239,0 +25240,0 +25241,0 +25242,0 +25243,0 +25244,0 +25245,0 +25246,0 +25247,0 +25248,0 +25249,0 +25250,0 +25251,0 +25252,0 +25253,0 +25254,0 +25255,0 +25256,0 +25257,0 +25258,0 +25259,0 +25260,0 +25261,0 +25262,0 +25263,0 +25264,0 +25265,0 +25266,0 +25267,0 +25268,0 +25269,0 +25270,0 +25271,0 +25272,0 +25273,0 +25274,0 +25275,0 +25276,0 +25277,0 +25278,0 +25279,0 +25280,0 +25281,0 +25282,0 +25283,0 +25284,0 +25285,0 +25286,0 +25287,0 +25288,0 +25289,0 +25290,0 +25291,0 +25292,0 +25293,0 +25294,0 +25295,0 +25296,0 +25297,0 +25298,0 +25299,0 +25300,0 +25301,0 +25302,0 +25303,0 +25304,0 +25305,0 +25306,0 +25307,0 +25308,0 +25309,0 +25310,0 +25311,0 +25312,0 +25313,0 +25314,0 +25315,0 +25316,0 +25317,0 +25318,0 +25319,0 +25320,0 +25321,0 +25322,0 +25323,0 +25324,0 +25325,0 +25326,0 +25327,0 +25328,0 +25329,0 +25330,0 +25331,0 +25332,0 +25333,0 +25334,0 +25335,0 +25336,0 +25337,0 +25338,0 +25339,0 +25340,0 +25341,0 +25342,0 +25343,0 +25344,0 +25345,0 +25346,0 +25347,0 +25348,0 +25349,0 +25350,0 +25351,0 +25352,0 +25353,0 +25354,0 +25355,0 +25356,0 +25357,0 +25358,0 +25359,0 +25360,0 +25361,0 +25362,0 +25363,0 +25364,0 +25365,0 +25366,0 +25367,0 +25368,0 +25369,0 +25370,0 +25371,0 +25372,0 +25373,0 +25374,0 +25375,0 +25376,0 +25377,0 +25378,0 +25379,0 +25380,0 +25381,0 +25382,0 +25383,0 +25384,0 +25385,0 +25386,0 +25387,0 +25388,0 +25389,0 +25390,0 +25391,0 +25392,0 +25393,0 +25394,0 +25395,0 +25396,0 +25397,0 +25398,0 +25399,0 +25400,0 +25401,0 +25402,0 +25403,0 +25404,0 +25405,0 +25406,0 +25407,0 +25408,0 +25409,0 +25410,0 +25411,0 +25412,0 +25413,0 +25414,0 +25415,0 +25416,0 +25417,0 +25418,0 +25419,0 +25420,0 +25421,0 +25422,0 +25423,0 +25424,0 +25425,0 +25426,0 +25427,0 +25428,0 +25429,0 +25430,0 +25431,0 +25432,0 +25433,0 +25434,0 +25435,0 +25436,0 +25437,0 +25438,0 +25439,0 +25440,0 +25441,0 +25442,0 +25443,0 +25444,0 +25445,0 +25446,0 +25447,0 +25448,0 +25449,0 +25450,0 +25451,0 +25452,0 +25453,0 +25454,0 +25455,0 +25456,0 +25457,0 +25458,0 +25459,0 +25460,0 +25461,0 +25462,0 +25463,0 +25464,0 +25465,0 +25466,0 +25467,0 +25468,0 +25469,0 +25470,0 +25471,0 +25472,0 +25473,0 +25474,0 +25475,0 +25476,0 +25477,0 +25478,0 +25479,0 +25480,0 +25481,0 +25482,0 +25483,0 +25484,0 +25485,0 +25486,0 +25487,0 +25488,0 +25489,0 +25490,0 +25491,0 +25492,0 +25493,0 +25494,0 +25495,0 +25496,0 +25497,0 +25498,0 +25499,0 +25500,0 +25501,0 +25502,0 +25503,0 +25504,0 +25505,0 +25506,0 +25507,0 +25508,0 +25509,0 +25510,0 +25511,0 +25512,0 +25513,0 +25514,0 +25515,0 +25516,0 +25517,0 +25518,0 +25519,0 +25520,0 +25521,0 +25522,0 +25523,0 +25524,0 +25525,0 +25526,0 +25527,0 +25528,0 +25529,0 +25530,0 +25531,0 +25532,0 +25533,0 +25534,0 +25535,0 +25536,0 +25537,0 +25538,0 +25539,0 +25540,0 +25541,0 +25542,0 +25543,0 +25544,0 +25545,0 +25546,0 +25547,0 +25548,0 +25549,0 +25550,0 +25551,0 +25552,0 +25553,0 +25554,0 +25555,0 +25556,0 +25557,0 +25558,0 +25559,0 +25560,0 +25561,0 +25562,0 +25563,0 +25564,0 +25565,0 +25566,0 +25567,0 +25568,0 +25569,0 +25570,0 +25571,0 +25572,0 +25573,0 +25574,0 +25575,0 +25576,0 +25577,0 +25578,0 +25579,0 +25580,0 +25581,0 +25582,0 +25583,0 +25584,0 +25585,0 +25586,0 +25587,0 +25588,0 +25589,0 +25590,0 +25591,0 +25592,0 +25593,0 +25594,0 +25595,0 +25596,0 +25597,0 +25598,0 +25599,0 +25600,0 +25601,0 +25602,0 +25603,0 +25604,0 +25605,0 +25606,0 +25607,0 +25608,0 +25609,0 +25610,0 +25611,0 +25612,0 +25613,0 +25614,0 +25615,0 +25616,0 +25617,0 +25618,0 +25619,0 +25620,0 +25621,0 +25622,0 +25623,0 +25624,0 +25625,0 +25626,0 +25627,0 +25628,0 +25629,0 +25630,0 +25631,0 +25632,0 +25633,0 +25634,0 +25635,0 +25636,0 +25637,0 +25638,0 +25639,0 +25640,0 +25641,0 +25642,0 +25643,0 +25644,0 +25645,0 +25646,0 +25647,0 +25648,0 +25649,0 +25650,0 +25651,0 +25652,0 +25653,0 +25654,0 +25655,0 +25656,0 +25657,0 +25658,0 +25659,0 +25660,0 +25661,0 +25662,0 +25663,0 +25664,0 +25665,0 +25666,0 +25667,0 +25668,0 +25669,0 +25670,0 +25671,0 +25672,0 +25673,0 +25674,0 +25675,0 +25676,0 +25677,0 +25678,0 +25679,0 +25680,0 +25681,0 +25682,0 +25683,0 +25684,0 +25685,0 +25686,0 +25687,0 +25688,0 +25689,0 +25690,0 +25691,0 +25692,0 +25693,0 +25694,0 +25695,0 +25696,0 +25697,0 +25698,0 +25699,0 +25700,0 +25701,0 +25702,0 +25703,0 +25704,0 +25705,0 +25706,0 +25707,0 +25708,0 +25709,0 +25710,0 +25711,0 +25712,0 +25713,0 +25714,0 +25715,0 +25716,0 +25717,0 +25718,0 +25719,0 +25720,0 +25721,0 +25722,0 +25723,0 +25724,0 +25725,0 +25726,0 +25727,0 +25728,0 +25729,0 +25730,0 +25731,0 +25732,0 +25733,0 +25734,0 +25735,0 +25736,0 +25737,0 +25738,0 +25739,0 +25740,0 +25741,0 +25742,0 +25743,0 +25744,0 +25745,0 +25746,0 +25747,0 +25748,0 +25749,0 +25750,0 +25751,0 +25752,0 +25753,0 +25754,0 +25755,0 +25756,0 +25757,0 +25758,0 +25759,0 +25760,0 +25761,0 +25762,0 +25763,0 +25764,0 +25765,0 +25766,0 +25767,0 +25768,0 +25769,0 +25770,0 +25771,0 +25772,0 +25773,0 +25774,0 +25775,0 +25776,0 +25777,0 +25778,0 +25779,0 +25780,0 +25781,0 +25782,0 +25783,0 +25784,0 +25785,0 +25786,0 +25787,0 +25788,0 +25789,0 +25790,0 +25791,0 +25792,0 +25793,0 +25794,0 +25795,0 +25796,0 +25797,0 +25798,0 +25799,0 +25800,0 +25801,0 +25802,0 +25803,0 +25804,0 +25805,0 +25806,0 +25807,0 +25808,0 +25809,0 +25810,0 +25811,0 +25812,0 +25813,0 +25814,0 +25815,0 +25816,0 +25817,0 +25818,0 +25819,0 +25820,0 +25821,0 +25822,0 +25823,0 +25824,0 +25825,0 +25826,0 +25827,0 +25828,0 +25829,0 +25830,0 +25831,0 +25832,0 +25833,0 +25834,0 +25835,0 +25836,0 +25837,0 +25838,0 +25839,0 +25840,0 +25841,0 +25842,0 +25843,0 +25844,0 +25845,0 +25846,0 +25847,0 +25848,0 +25849,0 +25850,0 +25851,0 +25852,0 +25853,0 +25854,0 +25855,0 +25856,0 +25857,0 +25858,0 +25859,0 +25860,0 +25861,0 +25862,0 +25863,0 +25864,0 +25865,0 +25866,0 +25867,0 +25868,0 +25869,0 +25870,0 +25871,0 +25872,0 +25873,0 +25874,0 +25875,0 +25876,0 +25877,0 +25878,0 +25879,0 +25880,0 +25881,0 +25882,0 +25883,0 +25884,0 +25885,0 +25886,0 +25887,0 +25888,0 +25889,0 +25890,0 +25891,0 +25892,0 +25893,0 +25894,0 +25895,0 +25896,0 +25897,0 +25898,0 +25899,0 +25900,0 +25901,0 +25902,0 +25903,0 +25904,0 +25905,0 +25906,0 +25907,0 +25908,0 +25909,0 +25910,0 +25911,0 +25912,0 +25913,0 +25914,0 +25915,0 +25916,0 +25917,0 +25918,0 +25919,0 +25920,0 +25921,0 +25922,0 +25923,0 +25924,0 +25925,0 +25926,0 +25927,0 +25928,0 +25929,0 +25930,0 +25931,0 +25932,0 +25933,0 +25934,0 +25935,0 +25936,0 +25937,0 +25938,0 +25939,0 +25940,0 +25941,0 +25942,0 +25943,0 +25944,0 +25945,0 +25946,0 +25947,0 +25948,0 +25949,0 +25950,0 +25951,0 +25952,0 +25953,0 +25954,0 +25955,0 +25956,0 +25957,0 +25958,0 +25959,0 +25960,0 +25961,0 +25962,0 +25963,0 +25964,0 +25965,0 +25966,0 +25967,0 +25968,0 +25969,0 +25970,0 +25971,0 +25972,0 +25973,0 +25974,0 +25975,0 +25976,0 +25977,0 +25978,0 +25979,0 +25980,0 +25981,0 +25982,0 +25983,0 +25984,0 +25985,0 +25986,0 +25987,0 +25988,0 +25989,0 +25990,0 +25991,0 +25992,0 +25993,0 +25994,0 +25995,0 +25996,0 +25997,0 +25998,0 +25999,0 +26000,0 +26001,0 +26002,0 +26003,0 +26004,0 +26005,0 +26006,0 +26007,0 +26008,0 +26009,0 +26010,0 +26011,0 +26012,0 +26013,0 +26014,0 +26015,0 +26016,0 +26017,0 +26018,0 +26019,0 +26020,0 +26021,0 +26022,0 +26023,0 +26024,0 +26025,0 +26026,0 +26027,0 +26028,0 +26029,0 +26030,0 +26031,0 +26032,0 +26033,0 +26034,0 +26035,0 +26036,0 +26037,0 +26038,0 +26039,0 +26040,0 +26041,0 +26042,0 +26043,0 +26044,0 +26045,0 +26046,0 +26047,0 +26048,0 +26049,0 +26050,0 +26051,0 +26052,0 +26053,0 +26054,0 +26055,0 +26056,0 +26057,0 +26058,0 +26059,0 +26060,0 +26061,0 +26062,0 +26063,0 +26064,0 +26065,0 +26066,0 +26067,0 +26068,0 +26069,0 +26070,0 +26071,0 +26072,0 +26073,0 +26074,0 +26075,0 +26076,0 +26077,0 +26078,0 +26079,0 +26080,0 +26081,0 +26082,0 +26083,0 +26084,0 +26085,0 +26086,0 +26087,0 +26088,0 +26089,0 +26090,0 +26091,0 +26092,0 +26093,0 +26094,0 +26095,0 +26096,0 +26097,0 +26098,0 +26099,0 +26100,0 +26101,0 +26102,0 +26103,0 +26104,0 +26105,0 +26106,0 +26107,0 +26108,0 +26109,0 +26110,0 +26111,0 +26112,0 +26113,0 +26114,0 +26115,0 +26116,0 +26117,0 +26118,0 +26119,0 +26120,0 +26121,0 +26122,0 +26123,0 +26124,0 +26125,0 +26126,0 +26127,0 +26128,0 +26129,0 +26130,0 +26131,0 +26132,0 +26133,0 +26134,0 +26135,0 +26136,0 +26137,0 +26138,0 +26139,0 +26140,0 +26141,0 +26142,0 +26143,0 +26144,0 +26145,0 +26146,0 +26147,0 +26148,0 +26149,0 +26150,0 +26151,0 +26152,0 +26153,0 +26154,0 +26155,0 +26156,0 +26157,0 +26158,0 +26159,0 +26160,0 +26161,0 +26162,0 +26163,0 +26164,0 +26165,0 +26166,0 +26167,0 +26168,0 +26169,0 +26170,0 +26171,0 +26172,0 +26173,0 +26174,0 +26175,0 +26176,0 +26177,0 +26178,0 +26179,0 +26180,0 +26181,0 +26182,0 +26183,0 +26184,0 +26185,0 +26186,0 +26187,0 +26188,0 +26189,0 +26190,0 +26191,0 +26192,0 +26193,0 +26194,0 +26195,0 +26196,0 +26197,0 +26198,0 +26199,0 +26200,0 +26201,0 +26202,0 +26203,0 +26204,0 +26205,0 +26206,0 +26207,0 +26208,0 +26209,0 +26210,0 +26211,0 +26212,0 +26213,0 +26214,0 +26215,0 +26216,0 +26217,0 +26218,0 +26219,0 +26220,0 +26221,0 +26222,0 +26223,0 +26224,0 +26225,0 +26226,0 +26227,0 +26228,0 +26229,0 +26230,0 +26231,0 +26232,0 +26233,0 +26234,0 +26235,0 +26236,0 +26237,0 +26238,0 +26239,0 +26240,0 +26241,0 +26242,0 +26243,0 +26244,0 +26245,0 +26246,0 +26247,0 +26248,0 +26249,0 +26250,0 +26251,0 +26252,0 +26253,0 +26254,0 +26255,0 +26256,0 +26257,0 +26258,0 +26259,0 +26260,0 +26261,0 +26262,0 +26263,0 +26264,0 +26265,0 +26266,0 +26267,0 +26268,0 +26269,0 +26270,0 +26271,0 +26272,0 +26273,0 +26274,0 +26275,0 +26276,0 +26277,0 +26278,0 +26279,0 +26280,0 +26281,0 +26282,0 +26283,0 +26284,0 +26285,0 +26286,0 +26287,0 +26288,0 +26289,0 +26290,0 +26291,0 +26292,0 +26293,0 +26294,0 +26295,0 +26296,0 +26297,0 +26298,0 +26299,0 +26300,0 +26301,0 +26302,0 +26303,0 +26304,0 +26305,0 +26306,0 +26307,0 +26308,0 +26309,0 +26310,0 +26311,0 +26312,0 +26313,0 +26314,0 +26315,0 +26316,0 +26317,0 +26318,0 +26319,0 +26320,0 +26321,0 +26322,0 +26323,0 +26324,0 +26325,0 +26326,0 +26327,0 +26328,0 +26329,0 +26330,0 +26331,0 +26332,0 +26333,0 +26334,0 +26335,0 +26336,0 +26337,0 +26338,0 +26339,0 +26340,0 +26341,0 +26342,0 +26343,0 +26344,0 +26345,0 +26346,0 +26347,0 +26348,0 +26349,0 +26350,0 +26351,0 +26352,0 +26353,0 +26354,0 +26355,0 +26356,0 +26357,0 +26358,0 +26359,0 +26360,0 +26361,0 +26362,0 +26363,0 +26364,0 +26365,0 +26366,0 +26367,0 +26368,0 +26369,0 +26370,0 +26371,0 +26372,0 +26373,0 +26374,0 +26375,0 +26376,0 +26377,0 +26378,0 +26379,0 +26380,0 +26381,0 +26382,0 +26383,0 +26384,0 +26385,0 +26386,0 +26387,0 +26388,0 +26389,0 +26390,0 +26391,0 +26392,0 +26393,0 +26394,0 +26395,0 +26396,0 +26397,0 +26398,0 +26399,0 +26400,0 +26401,0 +26402,0 +26403,0 +26404,0 +26405,0 +26406,0 +26407,0 +26408,0 +26409,0 +26410,0 +26411,0 +26412,0 +26413,0 +26414,0 +26415,0 +26416,0 +26417,0 +26418,0 +26419,0 +26420,0 +26421,0 +26422,0 +26423,0 +26424,0 +26425,0 +26426,0 +26427,0 +26428,0 +26429,0 +26430,0 +26431,0 +26432,0 +26433,0 +26434,0 +26435,0 +26436,0 +26437,0 +26438,0 +26439,0 +26440,0 +26441,0 +26442,0 +26443,0 +26444,0 +26445,0 +26446,0 +26447,0 +26448,0 +26449,0 +26450,0 +26451,0 +26452,0 +26453,0 +26454,0 +26455,0 +26456,0 +26457,0 +26458,0 +26459,0 +26460,0 +26461,0 +26462,0 +26463,0 +26464,0 +26465,0 +26466,0 +26467,0 +26468,0 +26469,0 +26470,0 +26471,0 +26472,0 +26473,0 +26474,0 +26475,0 +26476,0 +26477,0 +26478,0 +26479,0 +26480,0 +26481,0 +26482,0 +26483,0 +26484,0 +26485,0 +26486,0 +26487,0 +26488,0 +26489,0 +26490,0 +26491,0 +26492,0 +26493,0 +26494,0 +26495,0 +26496,0 +26497,0 +26498,0 +26499,0 +26500,0 +26501,0 +26502,0 +26503,0 +26504,0 +26505,0 +26506,0 +26507,0 +26508,0 +26509,0 +26510,0 +26511,0 +26512,0 +26513,0 +26514,0 +26515,0 +26516,0 +26517,0 +26518,0 +26519,0 +26520,0 +26521,0 +26522,0 +26523,0 +26524,0 +26525,0 +26526,0 +26527,0 +26528,0 +26529,0 +26530,0 +26531,0 +26532,0 +26533,0 +26534,0 +26535,0 +26536,0 +26537,0 +26538,0 +26539,0 +26540,0 +26541,0 +26542,0 +26543,0 +26544,0 +26545,0 +26546,0 +26547,0 +26548,0 +26549,0 +26550,0 +26551,0 +26552,0 +26553,0 +26554,0 +26555,0 +26556,0 +26557,0 +26558,0 +26559,0 +26560,0 +26561,0 +26562,0 +26563,0 +26564,0 +26565,0 +26566,0 +26567,0 +26568,0 +26569,0 +26570,0 +26571,0 +26572,0 +26573,0 +26574,0 +26575,0 +26576,0 +26577,0 +26578,0 +26579,0 +26580,0 +26581,0 +26582,0 +26583,0 +26584,0 +26585,0 +26586,0 +26587,0 +26588,0 +26589,0 +26590,0 +26591,0 +26592,0 +26593,0 +26594,0 +26595,0 +26596,0 +26597,0 +26598,0 +26599,0 +26600,0 +26601,0 +26602,0 +26603,0 +26604,0 +26605,0 +26606,0 +26607,0 +26608,0 +26609,0 +26610,0 +26611,0 +26612,0 +26613,0 +26614,0 +26615,0 +26616,0 +26617,0 +26618,0 +26619,0 +26620,0 +26621,0 +26622,0 +26623,0 +26624,0 +26625,0 +26626,0 +26627,0 +26628,0 +26629,0 +26630,0 +26631,0 +26632,0 +26633,0 +26634,0 +26635,0 +26636,0 +26637,0 +26638,0 +26639,0 +26640,0 +26641,0 +26642,0 +26643,0 +26644,0 +26645,0 +26646,0 +26647,0 +26648,0 +26649,0 +26650,0 +26651,0 +26652,0 +26653,0 +26654,0 +26655,0 +26656,0 +26657,0 +26658,0 +26659,0 +26660,0 +26661,0 +26662,0 +26663,0 +26664,0 +26665,0 +26666,0 +26667,0 +26668,0 +26669,0 +26670,0 +26671,0 +26672,0 +26673,0 +26674,0 +26675,0 +26676,0 +26677,0 +26678,0 +26679,0 +26680,0 +26681,0 +26682,0 +26683,0 +26684,0 +26685,0 +26686,0 +26687,0 +26688,0 +26689,0 +26690,0 +26691,0 +26692,0 +26693,0 +26694,0 +26695,0 +26696,0 +26697,0 +26698,0 +26699,0 +26700,0 +26701,0 +26702,0 +26703,0 +26704,0 +26705,0 +26706,0 +26707,0 +26708,0 +26709,0 +26710,0 +26711,0 +26712,0 +26713,0 +26714,0 +26715,0 +26716,0 +26717,0 +26718,0 +26719,0 +26720,0 +26721,0 +26722,0 +26723,0 +26724,0 +26725,0 +26726,0 +26727,0 +26728,0 +26729,0 +26730,0 +26731,0 +26732,0 +26733,0 +26734,0 +26735,0 +26736,0 +26737,0 +26738,0 +26739,0 +26740,0 +26741,0 +26742,0 +26743,0 +26744,0 +26745,0 +26746,0 +26747,0 +26748,0 +26749,0 +26750,0 +26751,0 +26752,0 +26753,0 +26754,0 +26755,0 +26756,0 +26757,0 +26758,0 +26759,0 +26760,0 +26761,0 +26762,0 +26763,0 +26764,0 +26765,0 +26766,0 +26767,0 +26768,0 +26769,0 +26770,0 +26771,0 +26772,0 +26773,0 +26774,0 +26775,0 +26776,0 +26777,0 +26778,0 +26779,0 +26780,0 +26781,0 +26782,0 +26783,0 +26784,0 +26785,0 +26786,0 +26787,0 +26788,0 +26789,0 +26790,0 +26791,0 +26792,0 +26793,0 +26794,0 +26795,0 +26796,0 +26797,0 +26798,0 +26799,0 +26800,0 +26801,0 +26802,0 +26803,0 +26804,0 +26805,0 +26806,0 +26807,0 +26808,0 +26809,0 +26810,0 +26811,0 +26812,0 +26813,0 +26814,0 +26815,0 +26816,0 +26817,0 +26818,0 +26819,0 +26820,0 +26821,0 +26822,0 +26823,0 +26824,0 +26825,0 +26826,0 +26827,0 +26828,0 +26829,0 +26830,0 +26831,0 +26832,0 +26833,0 +26834,0 +26835,0 +26836,0 +26837,0 +26838,0 +26839,0 +26840,0 +26841,0 +26842,0 +26843,0 +26844,0 +26845,0 +26846,0 +26847,0 +26848,0 +26849,0 +26850,0 +26851,0 +26852,0 +26853,0 +26854,0 +26855,0 +26856,0 +26857,0 +26858,0 +26859,0 +26860,0 +26861,0 +26862,0 +26863,0 +26864,0 +26865,0 +26866,0 +26867,0 +26868,0 +26869,0 +26870,0 +26871,0 +26872,0 +26873,0 +26874,0 +26875,0 +26876,0 +26877,0 +26878,0 +26879,0 +26880,0 +26881,0 +26882,0 +26883,0 +26884,0 +26885,0 +26886,0 +26887,0 +26888,0 +26889,0 +26890,0 +26891,0 +26892,0 +26893,0 +26894,0 +26895,0 +26896,0 +26897,0 +26898,0 +26899,0 +26900,0 +26901,0 +26902,0 +26903,0 +26904,0 +26905,0 +26906,0 +26907,0 +26908,0 +26909,0 +26910,0 +26911,0 +26912,0 +26913,0 +26914,0 +26915,0 +26916,0 +26917,0 +26918,0 +26919,0 +26920,0 +26921,0 +26922,0 +26923,0 +26924,0 +26925,0 +26926,0 +26927,0 +26928,0 +26929,0 +26930,0 +26931,0 +26932,0 +26933,0 +26934,0 +26935,0 +26936,0 +26937,0 +26938,0 +26939,0 +26940,0 +26941,0 +26942,0 +26943,0 +26944,0 +26945,0 +26946,0 +26947,0 +26948,0 +26949,0 +26950,0 +26951,0 +26952,0 +26953,0 +26954,0 +26955,0 +26956,0 +26957,0 +26958,0 +26959,0 +26960,0 +26961,0 +26962,0 +26963,0 +26964,0 +26965,0 +26966,0 +26967,0 +26968,0 +26969,0 +26970,0 +26971,0 +26972,0 +26973,0 +26974,0 +26975,0 +26976,0 +26977,0 +26978,0 +26979,0 +26980,0 +26981,0 +26982,0 +26983,0 +26984,0 +26985,0 +26986,0 +26987,0 +26988,0 +26989,0 +26990,0 +26991,0 +26992,0 +26993,0 +26994,0 +26995,0 +26996,0 +26997,0 +26998,0 +26999,0 +27000,0 +27001,0 +27002,0 +27003,0 +27004,0 +27005,0 +27006,0 +27007,0 +27008,0 +27009,0 +27010,0 +27011,0 +27012,0 +27013,0 +27014,0 +27015,0 +27016,0 +27017,0 +27018,0 +27019,0 +27020,0 +27021,0 +27022,0 +27023,0 +27024,0 +27025,0 +27026,0 +27027,0 +27028,0 +27029,0 +27030,0 +27031,0 +27032,0 +27033,0 +27034,0 +27035,0 +27036,0 +27037,0 +27038,0 +27039,0 +27040,0 +27041,0 +27042,0 +27043,0 +27044,0 +27045,0 +27046,0 +27047,0 +27048,0 +27049,0 +27050,0 +27051,0 +27052,0 +27053,0 +27054,0 +27055,0 +27056,0 +27057,0 +27058,0 +27059,0 +27060,0 +27061,0 +27062,0 +27063,0 +27064,0 +27065,0 +27066,0 +27067,0 +27068,0 +27069,0 +27070,0 +27071,0 +27072,0 +27073,0 +27074,0 +27075,0 +27076,0 +27077,0 +27078,0 +27079,0 +27080,0 +27081,0 +27082,0 +27083,0 +27084,0 +27085,0 +27086,0 +27087,0 +27088,0 +27089,0 +27090,0 +27091,0 +27092,0 +27093,0 +27094,0 +27095,0 +27096,0 +27097,0 +27098,0 +27099,0 +27100,0 +27101,0 +27102,0 +27103,0 +27104,0 +27105,0 +27106,0 +27107,0 +27108,0 +27109,0 +27110,0 +27111,0 +27112,0 +27113,0 +27114,0 +27115,0 +27116,0 +27117,0 +27118,0 +27119,0 +27120,0 +27121,0 +27122,0 +27123,0 +27124,0 +27125,0 +27126,0 +27127,0 +27128,0 +27129,0 +27130,0 +27131,0 +27132,0 +27133,0 +27134,0 +27135,0 +27136,0 +27137,0 +27138,0 +27139,0 +27140,0 +27141,0 +27142,0 +27143,0 +27144,0 +27145,0 +27146,0 +27147,0 +27148,0 +27149,0 +27150,0 +27151,0 +27152,0 +27153,0 +27154,0 +27155,0 +27156,0 +27157,0 +27158,0 +27159,0 +27160,0 +27161,0 +27162,0 +27163,0 +27164,0 +27165,0 +27166,0 +27167,0 +27168,0 +27169,0 +27170,0 +27171,0 +27172,0 +27173,0 +27174,0 +27175,0 +27176,0 +27177,0 +27178,0 +27179,0 +27180,0 +27181,0 +27182,0 +27183,0 +27184,0 +27185,0 +27186,0 +27187,0 +27188,0 +27189,0 +27190,0 +27191,0 +27192,0 +27193,0 +27194,0 +27195,0 +27196,0 +27197,0 +27198,0 +27199,0 +27200,0 +27201,0 +27202,0 +27203,0 +27204,0 +27205,0 +27206,0 +27207,0 +27208,0 +27209,0 +27210,0 +27211,0 +27212,0 +27213,0 +27214,0 +27215,0 +27216,0 +27217,0 +27218,0 +27219,0 +27220,0 +27221,0 +27222,0 +27223,0 +27224,0 +27225,0 +27226,0 +27227,0 +27228,0 +27229,0 +27230,0 +27231,0 +27232,0 +27233,0 +27234,0 +27235,0 +27236,0 +27237,0 +27238,0 +27239,0 +27240,0 +27241,0 +27242,0 +27243,0 +27244,0 +27245,0 +27246,0 +27247,0 +27248,0 +27249,0 +27250,0 +27251,0 +27252,0 +27253,0 +27254,0 +27255,0 +27256,0 +27257,0 +27258,0 +27259,0 +27260,0 +27261,0 +27262,0 +27263,0 +27264,0 +27265,0 +27266,0 +27267,0 +27268,0 +27269,0 +27270,0 +27271,0 +27272,0 +27273,0 +27274,0 +27275,0 +27276,0 +27277,0 +27278,0 +27279,0 +27280,0 +27281,0 +27282,0 +27283,0 +27284,0 +27285,0 +27286,0 +27287,0 +27288,0 +27289,0 +27290,0 +27291,0 +27292,0 +27293,0 +27294,0 +27295,0 +27296,0 +27297,0 +27298,0 +27299,0 +27300,0 +27301,0 +27302,0 +27303,0 +27304,0 +27305,0 +27306,0 +27307,0 +27308,0 +27309,0 +27310,0 +27311,0 +27312,0 +27313,0 +27314,0 +27315,0 +27316,0 +27317,0 +27318,0 +27319,0 +27320,0 +27321,0 +27322,0 +27323,0 +27324,0 +27325,0 +27326,0 +27327,0 +27328,0 +27329,0 +27330,0 +27331,0 +27332,0 +27333,0 +27334,0 +27335,0 +27336,0 +27337,0 +27338,0 +27339,0 +27340,0 +27341,0 +27342,0 +27343,0 +27344,0 +27345,0 +27346,0 +27347,0 +27348,0 +27349,0 +27350,0 +27351,0 +27352,0 +27353,0 +27354,0 +27355,0 +27356,0 +27357,0 +27358,0 +27359,0 +27360,0 +27361,0 +27362,0 +27363,0 +27364,0 +27365,0 +27366,0 +27367,0 +27368,0 +27369,0 +27370,0 +27371,0 +27372,0 +27373,0 +27374,0 +27375,0 +27376,0 +27377,0 +27378,0 +27379,0 +27380,0 +27381,0 +27382,0 +27383,0 +27384,0 +27385,0 +27386,0 +27387,0 +27388,0 +27389,0 +27390,0 +27391,0 +27392,0 +27393,0 +27394,0 +27395,0 +27396,0 +27397,0 +27398,0 +27399,0 +27400,0 +27401,0 +27402,0 +27403,0 +27404,0 +27405,0 +27406,0 +27407,0 +27408,0 +27409,0 +27410,0 +27411,0 +27412,0 +27413,0 +27414,0 +27415,0 +27416,0 +27417,0 +27418,0 +27419,0 +27420,0 +27421,0 +27422,0 +27423,0 +27424,0 +27425,0 +27426,0 +27427,0 +27428,0 +27429,0 +27430,0 +27431,0 +27432,0 +27433,0 +27434,0 +27435,0 +27436,0 +27437,0 +27438,0 +27439,0 +27440,0 +27441,0 +27442,0 +27443,0 +27444,0 +27445,0 +27446,0 +27447,0 +27448,0 +27449,0 +27450,0 +27451,0 +27452,0 +27453,0 +27454,0 +27455,0 +27456,0 +27457,0 +27458,0 +27459,0 +27460,0 +27461,0 +27462,0 +27463,0 +27464,0 +27465,0 +27466,0 +27467,0 +27468,0 +27469,0 +27470,0 +27471,0 +27472,0 +27473,0 +27474,0 +27475,0 +27476,0 +27477,0 +27478,0 +27479,0 +27480,0 +27481,0 +27482,0 +27483,0 +27484,0 +27485,0 +27486,0 +27487,0 +27488,0 +27489,0 +27490,0 +27491,0 +27492,0 +27493,0 +27494,0 +27495,0 +27496,0 +27497,0 +27498,0 +27499,0 +27500,0 +27501,0 +27502,0 +27503,0 +27504,0 +27505,0 +27506,0 +27507,0 +27508,0 +27509,0 +27510,0 +27511,0 +27512,0 +27513,0 +27514,0 +27515,0 +27516,0 +27517,0 +27518,0 +27519,0 +27520,0 +27521,0 +27522,0 +27523,0 +27524,0 +27525,0 +27526,0 +27527,0 +27528,0 +27529,0 +27530,0 +27531,0 +27532,0 +27533,0 +27534,0 +27535,0 +27536,0 +27537,0 +27538,0 +27539,0 +27540,0 +27541,0 +27542,0 +27543,0 +27544,0 +27545,0 +27546,0 +27547,0 +27548,0 +27549,0 +27550,0 +27551,0 +27552,0 +27553,0 +27554,0 +27555,0 +27556,0 +27557,0 +27558,0 +27559,0 +27560,0 +27561,0 +27562,0 +27563,0 +27564,0 +27565,0 +27566,0 +27567,0 +27568,0 +27569,0 +27570,0 +27571,0 +27572,0 +27573,0 +27574,0 +27575,0 +27576,0 +27577,0 +27578,0 +27579,0 +27580,0 +27581,0 +27582,0 +27583,0 +27584,0 +27585,0 +27586,0 +27587,0 +27588,0 +27589,0 +27590,0 +27591,0 +27592,0 +27593,0 +27594,0 +27595,0 +27596,0 +27597,0 +27598,0 +27599,0 +27600,0 +27601,0 +27602,0 +27603,0 +27604,0 +27605,0 +27606,0 +27607,0 +27608,0 +27609,0 +27610,0 +27611,0 +27612,0 +27613,0 +27614,0 +27615,0 +27616,0 +27617,0 +27618,0 +27619,0 +27620,0 +27621,0 +27622,0 +27623,0 +27624,0 +27625,0 +27626,0 +27627,0 +27628,0 +27629,0 +27630,0 +27631,0 +27632,0 +27633,0 +27634,0 +27635,0 +27636,0 +27637,0 +27638,0 +27639,0 +27640,0 +27641,0 +27642,0 +27643,0 +27644,0 +27645,0 +27646,0 +27647,0 +27648,0 +27649,0 +27650,0 +27651,0 +27652,0 +27653,0 +27654,0 +27655,0 +27656,0 +27657,0 +27658,0 +27659,0 +27660,0 +27661,0 +27662,0 +27663,0 +27664,0 +27665,0 +27666,0 +27667,0 +27668,0 +27669,0 +27670,0 +27671,0 +27672,0 +27673,0 +27674,0 +27675,0 +27676,0 +27677,0 +27678,0 +27679,0 +27680,0 +27681,0 +27682,0 +27683,0 +27684,0 +27685,0 +27686,0 +27687,0 +27688,0 +27689,0 +27690,0 +27691,0 +27692,0 +27693,0 +27694,0 +27695,0 +27696,0 +27697,0 +27698,0 +27699,0 +27700,0 +27701,0 +27702,0 +27703,0 +27704,0 +27705,0 +27706,0 +27707,0 +27708,0 +27709,0 +27710,0 +27711,0 +27712,0 +27713,0 +27714,0 +27715,0 +27716,0 +27717,0 +27718,0 +27719,0 +27720,0 +27721,0 +27722,0 +27723,0 +27724,0 +27725,0 +27726,0 +27727,0 +27728,0 +27729,0 +27730,0 +27731,0 +27732,0 +27733,0 +27734,0 +27735,0 +27736,0 +27737,0 +27738,0 +27739,0 +27740,0 +27741,0 +27742,0 +27743,0 +27744,0 +27745,0 +27746,0 +27747,0 +27748,0 +27749,0 +27750,0 +27751,0 +27752,0 +27753,0 +27754,0 +27755,0 +27756,0 +27757,0 +27758,0 +27759,0 +27760,0 +27761,0 +27762,0 +27763,0 +27764,0 +27765,0 +27766,0 +27767,0 +27768,0 +27769,0 +27770,0 +27771,0 +27772,0 +27773,0 +27774,0 +27775,0 +27776,0 +27777,0 +27778,0 +27779,0 +27780,0 +27781,0 +27782,0 +27783,0 +27784,0 +27785,0 +27786,0 +27787,0 +27788,0 +27789,0 +27790,0 +27791,0 +27792,0 +27793,0 +27794,0 +27795,0 +27796,0 +27797,0 +27798,0 +27799,0 +27800,0 +27801,0 +27802,0 +27803,0 +27804,0 +27805,0 +27806,0 +27807,0 +27808,0 +27809,0 +27810,0 +27811,0 +27812,0 +27813,0 +27814,0 +27815,0 +27816,0 +27817,0 +27818,0 +27819,0 +27820,0 +27821,0 +27822,0 +27823,0 +27824,0 +27825,0 +27826,0 +27827,0 +27828,0 +27829,0 +27830,0 +27831,0 +27832,0 +27833,0 +27834,0 +27835,0 +27836,0 +27837,0 +27838,0 +27839,0 +27840,0 +27841,0 +27842,0 +27843,0 +27844,0 +27845,0 +27846,0 +27847,0 +27848,0 +27849,0 +27850,0 +27851,0 +27852,0 +27853,0 +27854,0 +27855,0 +27856,0 +27857,0 +27858,0 +27859,0 +27860,0 +27861,0 +27862,0 +27863,0 +27864,0 +27865,0 +27866,0 +27867,0 +27868,0 +27869,0 +27870,0 +27871,0 +27872,0 +27873,0 +27874,0 +27875,0 +27876,0 +27877,0 +27878,0 +27879,0 +27880,0 +27881,0 +27882,0 +27883,0 +27884,0 +27885,0 +27886,0 +27887,0 +27888,0 +27889,0 +27890,0 +27891,0 +27892,0 +27893,0 +27894,0 +27895,0 +27896,0 +27897,0 +27898,0 +27899,0 +27900,0 +27901,0 +27902,0 +27903,0 +27904,0 +27905,0 +27906,0 +27907,0 +27908,0 +27909,0 +27910,0 +27911,0 +27912,0 +27913,0 +27914,0 +27915,0 +27916,0 +27917,0 +27918,0 +27919,0 +27920,0 +27921,0 +27922,0 +27923,0 +27924,0 +27925,0 +27926,0 +27927,0 +27928,0 +27929,0 +27930,0 +27931,0 +27932,0 +27933,0 +27934,0 +27935,0 +27936,0 +27937,0 +27938,0 +27939,0 +27940,0 +27941,0 +27942,0 +27943,0 +27944,0 +27945,0 +27946,0 +27947,0 +27948,0 +27949,0 +27950,0 +27951,0 +27952,0 +27953,0 +27954,0 +27955,0 +27956,0 +27957,0 +27958,0 +27959,0 +27960,0 +27961,0 +27962,0 +27963,0 +27964,0 +27965,0 +27966,0 +27967,0 +27968,0 +27969,0 +27970,0 +27971,0 +27972,0 +27973,0 +27974,0 +27975,0 +27976,0 +27977,0 +27978,0 +27979,0 +27980,0 +27981,0 +27982,0 +27983,0 +27984,0 +27985,0 +27986,0 +27987,0 +27988,0 +27989,0 +27990,0 +27991,0 +27992,0 +27993,0 +27994,0 +27995,0 +27996,0 +27997,0 +27998,0 +27999,0 +28000,0 diff --git a/digit-recognition-kaggle-competition/requirements.txt b/digit-recognition-kaggle-competition/requirements.txt index 92f270bc..321b3578 100644 --- a/digit-recognition-kaggle-competition/requirements.txt +++ b/digit-recognition-kaggle-competition/requirements.txt @@ -1,4 +1,4 @@ -pandas==1.1.5 -seaborn==0.9.0 -tensorflow==2.3.0 -wget==3.2 +pandas==1.1.5 +seaborn==0.9.0 +tensorflow==2.3.0 +wget==3.2 diff --git a/facial-keypoints-detection-kaggle-competition/Readme.md b/facial-keypoints-detection-kaggle-competition/Readme.md index a283267c..055077c1 100644 --- a/facial-keypoints-detection-kaggle-competition/Readme.md +++ b/facial-keypoints-detection-kaggle-competition/Readme.md @@ -1,198 +1,198 @@ -# Objective - -This example is based on the Facial Keypoints Detection Kaggle competition. The objective of this exercise is to predict keypoint positions on face images. - -## Environment - -This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 based system which includes all Intel and AMD based CPU's. ARM based systems are not supported. - -## Prerequisites for Building the Kubeflow Pipeline - -### Kubeflow - -It is assumed that you have Kubeflow installed. - -### Docker - -Docker is used to create an image to run each component in the pipeline. - -### Kubeflow Pipelines - -Kubeflow Pipelines connects each Docker-based component to create a pipeline. Each pipeline is a reproducible workflow wherein we pass input arguments and run the entire workflow. - -# Apply PodDefault resource - -## Step 1: Generate Kaggle API token -The input data needed to run this tutorial is been pulled from Kaggle . In order to pull the data we need to create a Kaggle account , user needs to register with his email and password and create a Kaggle username. - -Once we have successfully registered our Kaggle account. Now, we have to access the API Token . API access is needed to pull data from Kaggle , to get the API access go to you Kaggle profile and click on your profile picture on the top right we will see this option: - -Account - -Select “Account” from the menu. - -Scroll down to the “API” section and click “Create New API Token” : -Screenshot 2022-05-10 at 1 03 34 PM - - -This will download a file ‘kaggle.json’ with the following contents : -``` -username “My username” -key “My key” -``` -Now, substitute your “username” for `` and your “key” for  `` and create a Kubernetes secret using: 
 -``` -kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= -``` - - -## Step2: Create a PodDefault resource - -We need a way to inject common data (env vars, volumes) to pods. In Kubeflow we use PodDefault resource which serves this usecase (reference: https://github.com/kubeflow/kubeflow/blob/master/components/admission-webhook/README.md). Using the PodDefault resource we can attach a secret to our data pulling step container which downloads data using Kaggle API. We create and apply PodDefault resource as follows : - -Create a `resource.yaml` file with the following code: - -``` -apiVersion: "kubeflow.org/v1alpha1" -kind: PodDefault -metadata: - name: kaggle-access -spec: - selector: - matchLabels: - kaggle-secret: "true" - desc: "kaggle-access" - volumeMounts: - - name: secret-volume - mountPath: /secret/kaggle - volumes: - - name: secret-volume - secret: - secretName: kaggle-secret -``` - -Apply the yaml with the following command: -``` -kubectl apply -f resource.yaml -``` - -# Build the Train and Evaluate images with Docker - -Kubeflow relies on Docker images to create pipelines. These images are pushed to a Docker container registry, from which Kubeflow accesses them. For the purposes of this how-to we are going to use Docker Hub as our registry. - -## Step 1: Log into Docker - -Start by creating a Docker account on DockerHub (https://hub.docker.com/). After signing up, Install Docker https://docs.docker.com/get-docker/ and enter `docker login` command on your terminal and enter your docker-hub username and password to log into Docker. - -## Step 2: Build the Train image - -Create a new build enviornment which contains Docker installed as highlighted in step1. After this in your new enviornment, on your terminal navigate to the pipeline-components/train/ directory and build the train Docker image using: -``` -$ cd pipeline-components/train/ -$ docker build -t /: . -``` -For example: -``` -$ docker build -t hubdocker76/demotrain:v8 . -``` -After building push the image using: -``` -$ docker push hubdocker76/demotrain:v8 -``` -## Step 3: Build the Evaluate image - -Next, on your docker enviornment go to terminal and navigate to the pipeline-components/eval/ directory and build the evaluate Docker image using: -``` -$ cd pipeline-components/eval/ -$ docker build -t /: . -``` -For example: -``` -$ docker build -t hubdocker76/demoeval:v3 . -``` -After building push the image using: -``` -$ docker push hubdocker76/demoeval:v3 -``` -## Kubeflow Pipeline - -As a needed step we create a virtual enviornment, that contains all components we need to convert our python code to yaml file. - -Steps to build a python virtual enviornment: - -Step a) Update pip -``` -python3 -m pip install --upgrade pip -``` - -Step b) Install virtualenv -``` -sudo pip3 install virtualenv -``` - -Step c) Check the installed version of venv -``` -virtualenv --version -``` - -Step d) Name your virtual enviornment as kfp -``` -virtualenv kfp -``` - -Step e) Activate your venv. -``` -source kfp/bin/activate -``` - -After this virtual environment will get activated. Now in our activated venv we need to install following packages: -``` -sudo apt-get update -sudo apt-get upgrade -sudo apt-get install -y git python3-pip - -python3 -m pip install kfp==1.1.2 -``` - -After installing packages create the yaml file - -Inside venv point your terminal to a path which contains our kfp file to build pipeline (facial-keypoints-detection-kfp.py) and run these commands: -``` -$ python3 facial-keypoints-detection-kfp.py -``` -…this will generate a yaml file: -``` -facial-keypoints-detection-kfp.py.yaml -``` - -## Run the Kubeflow Pipeline - -This `facial-keypoints-detection-kfp.py.yaml` file can then be uploaded to Kubeflow Pipelines UI from which you can create a Pipeline Run. The same yaml file will also be generated if we run the facial-keypoints-detection-kfp.ipynb notebook in the Notebook Server UI. - - -Upload file : -Screenshot 2022-05-23 at 10 08 40 PM - - -Create Run by selecting pipeline and give this run a name. Enter integer values for trial epoch and patience: -image - -Ideal values for trial epoch and patience are: trial=5, epoch=8, patience=3 - - -And then Start the Run. -Screenshot 2022-05-23 at 10 10 49 PM - - -# Kubeflow Pipeline with Kale - -To run this pipeline using the Kale JupyterLab extension, upload the `facial-keypoints-detection-kale.ipynb` file to your Kubeflow deployment where Kale is enabled. Once uploaded as a necessary step run the cell annotated with `skip` tag with function `download_kaggle_dataset`: - -Screenshot 2022-06-02 at 12 21 20 AM - - -This downloads the data using Kaggle API (use `` and `` as highlighted in `Apply PodDefault resource` step to get data using Kaggle API) this saves the download data to `my_data` folder. - -Only after the data is downloaded then click “compile and run” to create a pipeline run. - - +# Objective + +This example is based on the Facial Keypoints Detection Kaggle competition. The objective of this exercise is to predict keypoint positions on face images. + +## Environment + +This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 based system which includes all Intel and AMD based CPU's. ARM based systems are not supported. + +## Prerequisites for Building the Kubeflow Pipeline + +### Kubeflow + +It is assumed that you have Kubeflow installed. + +### Docker + +Docker is used to create an image to run each component in the pipeline. + +### Kubeflow Pipelines + +Kubeflow Pipelines connects each Docker-based component to create a pipeline. Each pipeline is a reproducible workflow wherein we pass input arguments and run the entire workflow. + +# Apply PodDefault resource + +## Step 1: Generate Kaggle API token +The input data needed to run this tutorial is been pulled from Kaggle . In order to pull the data we need to create a Kaggle account , user needs to register with his email and password and create a Kaggle username. + +Once we have successfully registered our Kaggle account. Now, we have to access the API Token . API access is needed to pull data from Kaggle , to get the API access go to you Kaggle profile and click on your profile picture on the top right we will see this option: + +Account + +Select “Account” from the menu. + +Scroll down to the “API” section and click “Create New API Token” : +Screenshot 2022-05-10 at 1 03 34 PM + + +This will download a file ‘kaggle.json’ with the following contents : +``` +username “My username” +key “My key” +``` +Now, substitute your “username” for `` and your “key” for  `` and create a Kubernetes secret using: 
 +``` +kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= +``` + + +## Step2: Create a PodDefault resource + +We need a way to inject common data (env vars, volumes) to pods. In Kubeflow we use PodDefault resource which serves this usecase (reference: https://github.com/kubeflow/kubeflow/blob/master/components/admission-webhook/README.md). Using the PodDefault resource we can attach a secret to our data pulling step container which downloads data using Kaggle API. We create and apply PodDefault resource as follows : + +Create a `resource.yaml` file with the following code: + +``` +apiVersion: "kubeflow.org/v1alpha1" +kind: PodDefault +metadata: + name: kaggle-access +spec: + selector: + matchLabels: + kaggle-secret: "true" + desc: "kaggle-access" + volumeMounts: + - name: secret-volume + mountPath: /secret/kaggle + volumes: + - name: secret-volume + secret: + secretName: kaggle-secret +``` + +Apply the yaml with the following command: +``` +kubectl apply -f resource.yaml +``` + +# Build the Train and Evaluate images with Docker + +Kubeflow relies on Docker images to create pipelines. These images are pushed to a Docker container registry, from which Kubeflow accesses them. For the purposes of this how-to we are going to use Docker Hub as our registry. + +## Step 1: Log into Docker + +Start by creating a Docker account on DockerHub (https://hub.docker.com/). After signing up, Install Docker https://docs.docker.com/get-docker/ and enter `docker login` command on your terminal and enter your docker-hub username and password to log into Docker. + +## Step 2: Build the Train image + +Create a new build enviornment which contains Docker installed as highlighted in step1. After this in your new enviornment, on your terminal navigate to the pipeline-components/train/ directory and build the train Docker image using: +``` +$ cd pipeline-components/train/ +$ docker build -t /: . +``` +For example: +``` +$ docker build -t hubdocker76/demotrain:v8 . +``` +After building push the image using: +``` +$ docker push hubdocker76/demotrain:v8 +``` +## Step 3: Build the Evaluate image + +Next, on your docker enviornment go to terminal and navigate to the pipeline-components/eval/ directory and build the evaluate Docker image using: +``` +$ cd pipeline-components/eval/ +$ docker build -t /: . +``` +For example: +``` +$ docker build -t hubdocker76/demoeval:v3 . +``` +After building push the image using: +``` +$ docker push hubdocker76/demoeval:v3 +``` +## Kubeflow Pipeline + +As a needed step we create a virtual enviornment, that contains all components we need to convert our python code to yaml file. + +Steps to build a python virtual enviornment: + +Step a) Update pip +``` +python3 -m pip install --upgrade pip +``` + +Step b) Install virtualenv +``` +sudo pip3 install virtualenv +``` + +Step c) Check the installed version of venv +``` +virtualenv --version +``` + +Step d) Name your virtual enviornment as kfp +``` +virtualenv kfp +``` + +Step e) Activate your venv. +``` +source kfp/bin/activate +``` + +After this virtual environment will get activated. Now in our activated venv we need to install following packages: +``` +sudo apt-get update +sudo apt-get upgrade +sudo apt-get install -y git python3-pip + +python3 -m pip install kfp==1.1.2 +``` + +After installing packages create the yaml file + +Inside venv point your terminal to a path which contains our kfp file to build pipeline (facial-keypoints-detection-kfp.py) and run these commands: +``` +$ python3 facial-keypoints-detection-kfp.py +``` +…this will generate a yaml file: +``` +facial-keypoints-detection-kfp.py.yaml +``` + +## Run the Kubeflow Pipeline + +This `facial-keypoints-detection-kfp.py.yaml` file can then be uploaded to Kubeflow Pipelines UI from which you can create a Pipeline Run. The same yaml file will also be generated if we run the facial-keypoints-detection-kfp.ipynb notebook in the Notebook Server UI. + + +Upload file : +Screenshot 2022-05-23 at 10 08 40 PM + + +Create Run by selecting pipeline and give this run a name. Enter integer values for trial epoch and patience: +image + +Ideal values for trial epoch and patience are: trial=5, epoch=8, patience=3 + + +And then Start the Run. +Screenshot 2022-05-23 at 10 10 49 PM + + +# Kubeflow Pipeline with Kale + +To run this pipeline using the Kale JupyterLab extension, upload the `facial-keypoints-detection-kale.ipynb` file to your Kubeflow deployment where Kale is enabled. Once uploaded as a necessary step run the cell annotated with `skip` tag with function `download_kaggle_dataset`: + +Screenshot 2022-06-02 at 12 21 20 AM + + +This downloads the data using Kaggle API (use `` and `` as highlighted in `Apply PodDefault resource` step to get data using Kaggle API) this saves the download data to `my_data` folder. + +Only after the data is downloaded then click “compile and run” to create a pipeline run. + + diff --git a/facial-keypoints-detection-kaggle-competition/facial-keypoints-detection-kfp.py b/facial-keypoints-detection-kaggle-competition/facial-keypoints-detection-kfp.py index 25834851..b980763d 100644 --- a/facial-keypoints-detection-kaggle-competition/facial-keypoints-detection-kfp.py +++ b/facial-keypoints-detection-kaggle-competition/facial-keypoints-detection-kfp.py @@ -1,45 +1,45 @@ -import kfp -from kfp import dsl - - -def SendMsg(trial, epoch, patience): - vop = dsl.VolumeOp(name="pvc", - resource_name="pvc", size='5Gi', - modes=dsl.VOLUME_MODE_RWO) - - return dsl.ContainerOp( - name = 'Train', - image = 'hubdocker76/demotrain:v8', # use this prebuilt image or replace image with your own custom image - command = ['python3', 'train.py'], - arguments=[ - '--trial', trial, - '--epoch', epoch, - '--patience', patience - ], - pvolumes={ - '/data': vop.volume - } - ) - -def GetMsg(comp1): - return dsl.ContainerOp( - name = 'Evaluate', - image = 'hubdocker76/demoeval:v3', # use this prebuilt image or replace image with your own custom image - pvolumes={ - '/data': comp1.pvolumes['/data'] - }, - command = ['python3', 'eval.py'] - ) - -@dsl.pipeline( - name = 'face pipeline', - description = 'pipeline to detect facial landmarks') -def passing_parameter(trial, epoch, patience): - comp1 = SendMsg(trial, epoch, patience).add_pod_label("kaggle-secret", "true") - comp2 = GetMsg(comp1) - -if __name__ == '__main__': - import kfp.compiler as compiler - compiler.Compiler().compile(passing_parameter, __file__ + '.yaml') - - +import kfp +from kfp import dsl + + +def SendMsg(trial, epoch, patience): + vop = dsl.VolumeOp(name="pvc", + resource_name="pvc", size='5Gi', + modes=dsl.VOLUME_MODE_RWO) + + return dsl.ContainerOp( + name = 'Train', + image = 'hubdocker76/demotrain:v8', # use this prebuilt image or replace image with your own custom image + command = ['python3', 'train.py'], + arguments=[ + '--trial', trial, + '--epoch', epoch, + '--patience', patience + ], + pvolumes={ + '/data': vop.volume + } + ) + +def GetMsg(comp1): + return dsl.ContainerOp( + name = 'Evaluate', + image = 'hubdocker76/demoeval:v3', # use this prebuilt image or replace image with your own custom image + pvolumes={ + '/data': comp1.pvolumes['/data'] + }, + command = ['python3', 'eval.py'] + ) + +@dsl.pipeline( + name = 'face pipeline', + description = 'pipeline to detect facial landmarks') +def passing_parameter(trial, epoch, patience): + comp1 = SendMsg(trial, epoch, patience).add_pod_label("kaggle-secret", "true") + comp2 = GetMsg(comp1) + +if __name__ == '__main__': + import kfp.compiler as compiler + compiler.Compiler().compile(passing_parameter, __file__ + '.yaml') + + diff --git a/github_issue_summarization/Makefile b/github_issue_summarization/Makefile old mode 100755 new mode 100644 diff --git a/github_issue_summarization/pipelines/components/kubeflow-resources/containers/tf-serving-gh/build.sh b/github_issue_summarization/pipelines/components/kubeflow-resources/containers/tf-serving-gh/build.sh old mode 100755 new mode 100644 diff --git a/github_issue_summarization/pipelines/components/t2t/containers/base/build.sh b/github_issue_summarization/pipelines/components/t2t/containers/base/build.sh old mode 100755 new mode 100644 diff --git a/github_issue_summarization/pipelines/components/t2t/containers/metadata-logger/build.sh b/github_issue_summarization/pipelines/components/t2t/containers/metadata-logger/build.sh old mode 100755 new mode 100644 diff --git a/github_issue_summarization/pipelines/components/t2t/containers/t2t_app/build.sh b/github_issue_summarization/pipelines/components/t2t/containers/t2t_app/build.sh old mode 100755 new mode 100644 diff --git a/github_issue_summarization/pipelines/components/t2t/containers/t2t_proc/build.sh b/github_issue_summarization/pipelines/components/t2t/containers/t2t_proc/build.sh old mode 100755 new mode 100644 diff --git a/github_issue_summarization/pipelines/components/t2t/containers/t2t_train/build.sh b/github_issue_summarization/pipelines/components/t2t/containers/t2t_train/build.sh old mode 100755 new mode 100644 diff --git a/github_issue_summarization/pipelines/components/t2t/containers/webapp-launcher/build.sh b/github_issue_summarization/pipelines/components/t2t/containers/webapp-launcher/build.sh old mode 100755 new mode 100644 diff --git a/h-and-m-fash-rec-kaggle-competition/README.md b/h-and-m-fash-rec-kaggle-competition/README.md index 82a62d1c..a8cfe169 100644 --- a/h-and-m-fash-rec-kaggle-competition/README.md +++ b/h-and-m-fash-rec-kaggle-competition/README.md @@ -1,179 +1,179 @@ -# Kaggle Featured Prediction Competition: H&M Personalized Fashion Recommendations - -In this [competition](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations), product recommendations have to be done based on previous purchases. There's a whole range of data available including customer meta data, product meta data, and meta data that spans from simple data, such as garment type and customer age, to text data from product descriptions, to image data from garment images. - -In this notebook we will be working with implicit's ALS library for our recommender systems. Please do check out the [docs](https://benfred.github.io/implicit/index.html) for more information. - -## Prerequisites for Building the Kubeflow Pipeline - -If you don’t already have Kubeflow up and running, we recommend signing up for a free trial of Arrikto's [Kubeflow as a Service](https://www.arrikto.com/kubeflow-as-a-service/). For the following example, we are using Kubeflow as a Service, but you should be able to run this example on any Kubeflow distribution. - -## Testing environment - -| Name | version | -| ------------- |:-------------:| -| Kubeflow | v1.4 | -| kfp | 1.8.11 | -| kubeflow-kale | 0.6.0 | - -## Initial Steps - -1. Please follow the Prerequisites section to get Kubeflow running. -2. Create a new Jupyter Notebook server with following resources - - CPU : 1 - - RAM : 32GB - - Workspace Volume : 50GB -3. Once you have the Jupyter Notebook server running, connect to it. -4. Clone this repo from the Terminal, so you have access to this directory. -5. Now before heading to Vanilla KFP steps, we need to save our Kaggle API credentials as a secret so that we can use the Kaggle Public [API](https://github.com/Kaggle/kaggle-api/blob/master/kaggle/api/kaggle_api_extended.py) to download the files from the Kaggle competition for our KFP/Kale pipeline. Following are the steps: - - If you are not a Kaggle user, you will first need to create a Kaggle account. After creation of the account, go to your Kaggle Account page and scroll down to API section. - -

- -

- - - Click on Create New API Token. A new API token in the form of kaggle.json file will be created which you can save locally. The kaggle.json file contains your Kaggle username and key. - - Once you have the API credentials, run the following command in the terminal with the username and key from the kaggle.json file that you just saved. - - ``` - kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= - - ``` - This creates a secret for our credentials which can then be mounted on our pods. - - - Next create a yaml file with the following code in it. This would then be used to create a pod-default resource to mount the secret to any pod with a specific label(in our case kaggle-secret =true) - - ``` - apiVersion: "kubeflow.org/v1alpha1" - kind: PodDefault - metadata: - name: kaggle-access - spec: - selector: - matchLabels: - kaggle-secret: "true" - desc: "kaggle-access" - volumeMounts: - - name: secret-volume - mountPath: /secret/kaggle - volumes: - - name: secret-volume - secret: - secretName: kaggle-secret - - ``` - - To create a pod-default resource, run the following command, - - ``` - kubectl apply -f - - ``` - You can check out the following [link](https://support.arrikto.com/hc/en-us/articles/6335158153489-Acessing-External-System-with-User-Credentials-Kaggle-Example-) for more details about accessing external system with user credentials. -6. With the completion of 5th step, you are good to start with Vanilla KFP steps. - - - -## Vanilla KFP version - -To start building out a Kubeflow pipeline, you need to get yourself acquainted with the Kubeflow Pipelines [documentation](https://www.kubeflow.org/docs/components/pipelines/sdk/build-pipeline/) to understand what the pipelines are, its components, what goes into these components. There are different ways to build out a pipeline component as mentioned [here](https://www.kubeflow.org/docs/components/pipelines/sdk/build-pipeline/#building-pipeline-components). In the following example, we are going to use the [lightweight python functions](https://www.kubeflow.org/docs/components/pipelines/sdk/python-function-components/) based components for building up the pipeline. - -### Step 1: Install the Kubeflow Pipeline SDK and import the required kfp packages to run the pipeline - -From kfp, we will be using [func_to_container_op](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.func_to_container_op) which would help in building the factory function from the python function and we will use [InputPath](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.InputPath) and [OutputPath](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.OutputPath) from the components package to pass the paths of the files or models to these tasks. The [passing of data](https://www.kubeflow.org/docs/components/pipelines/sdk/python-function-components/#pass-data) is being implemented by kfp’s supported data passing mechanism. InputPath and OutputPath is how you pass on the data or model between the components. For [passing values](https://www.kubeflow.org/docs/components/pipelines/sdk/python-function-components/#passing-parameters-by-value), we are using NamedTuples which allows us to send multiple values between components. - -### Step 2: Next build out the pipeline components - -Our Kubeflow pipeline is broken down into five pipeline components: - -- Download the data from Kaggle -- Load and Preprocess the data -- Creating Sparse Matrix -- Train data -- Predictions - -We convert each python function to a factory function using the func_to_container_op which will then be converted to a pipeline task for our pipeline function. - -### Step 3 : Creating pipeline function - -After building all the pipeline components, we have to define a pipeline function connecting all the pipeline components with appropriate inputs and outputs. This when run would generate the pipeline graph. - -Pipeline function: - -

- -

- - -### Step 4 : Running the pipeline using the kfp.client instance - -There are different ways to run the pipeline function as mentioned in the [documentation](https://www.kubeflow.org/docs/components/pipelines/sdk/build-pipeline/#compile-and-run-your-pipeline). We would run the pipeline using the Kubeflow Pipelines SDK client. - -

- -

- -Once all the cells are executed successfully, you should see two hyperlinks ‘Experiment details’ and ‘Run details’. Click on ‘Run details’ link to observe the pipeline running. - -The final pipeline graph would look as follow: - -

- -

- -## Kale KFP version - -For the Kaggle notebook example, we are using [Kubeflow as a Service](https://www.arrikto.com/kubeflow-as-a-service/). If you are using Kubeflow as a Service then Kale comes preinstalled. For users with a different Kubeflow setup, you can refer to the [GitHub link](https://github.com/kubeflow-kale/kale#getting-started) for installing the Kale JupyterLab extension on your setup. - -### Step 1: Install all the required packages - -Run the first code cell to install all the required packages (not available under the standard python library) by using the requirements.txt file. Restart the kernel after installation. - -### Step 2: Download the data from Kaggle - -Run the second code cell to download the relevant data from Kaggle using the Kaggle Public API. You will require the API credentials from the kaggle.json file you got earlier in the Initial Steps. For the Kale notebook version, you don't have to create the secret, just need the API credentials to download the data. Once the code cell is run, you should see a new "data" directory being created with the zip files downloaded and unzipped. Please ensure that you run the cell only once so you don't create nested directories. Restart the kernel before running the code cell again. - -### Step 3: Annotate the notebook with Kale tags - -The Kale notebook in the directory is already annotated. To see the annotations, open up the Kale Deployment panel and click on the Enable switch button. Once you have it switched on, you should see the following: - -

- -

- -Please take time to understand how each cell is annotated by clicking on the cell and checking out the tag being used and what are is its dependencies. Kale provides us with six tags for annotations: - -- Imports -- Functions -- Pipeline Parameters -- Pipeline Metrics -- Pipeline Step -- Skip Cell - -You can also see the tags being created by checking out the Cell Metadata by clicking on the Property Inspector above the Kale Deployment Panel button. - -

- -

- -### Step 2: Run the Kubeflow Pipeline - -Once you’ve tagged your notebook, click on the “Compile and Run” button in the Kale widget. Kale will perform the following tasks for you: - -- Validate the notebook -- Take a snapshot -- Compile the notebook -- Upload the pipeline -- Run the pipeline - -In the “Running pipeline” output, click on the “View” hyperlink. This will take you directly to the runtime execution graph where you can watch your pipeline execute and update in real-time. - -

- -

- -## Note: -Both notebooks have been tested out and the whole pipeline run for both the Vanilla KFP and the Kale KFP versions take around 2hrs. Most of the time is being consumed in the predictions pipeline stage. In case of any error, please test out with the following docker image. - -Notebook server docker image used: gcr.io/arrikto/jupyter-kale-py36@sha256:dd3f92ca66b46d247e4b9b6a9d84ffbb368646263c2e3909473c3b851f3fe198 - -If the error persists, please raise an issue. +# Kaggle Featured Prediction Competition: H&M Personalized Fashion Recommendations + +In this [competition](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations), product recommendations have to be done based on previous purchases. There's a whole range of data available including customer meta data, product meta data, and meta data that spans from simple data, such as garment type and customer age, to text data from product descriptions, to image data from garment images. + +In this notebook we will be working with implicit's ALS library for our recommender systems. Please do check out the [docs](https://benfred.github.io/implicit/index.html) for more information. + +## Prerequisites for Building the Kubeflow Pipeline + +If you don’t already have Kubeflow up and running, we recommend signing up for a free trial of Arrikto's [Kubeflow as a Service](https://www.arrikto.com/kubeflow-as-a-service/). For the following example, we are using Kubeflow as a Service, but you should be able to run this example on any Kubeflow distribution. + +## Testing environment + +| Name | version | +| ------------- |:-------------:| +| Kubeflow | v1.4 | +| kfp | 1.8.11 | +| kubeflow-kale | 0.6.0 | + +## Initial Steps + +1. Please follow the Prerequisites section to get Kubeflow running. +2. Create a new Jupyter Notebook server with following resources + - CPU : 1 + - RAM : 32GB + - Workspace Volume : 50GB +3. Once you have the Jupyter Notebook server running, connect to it. +4. Clone this repo from the Terminal, so you have access to this directory. +5. Now before heading to Vanilla KFP steps, we need to save our Kaggle API credentials as a secret so that we can use the Kaggle Public [API](https://github.com/Kaggle/kaggle-api/blob/master/kaggle/api/kaggle_api_extended.py) to download the files from the Kaggle competition for our KFP/Kale pipeline. Following are the steps: + - If you are not a Kaggle user, you will first need to create a Kaggle account. After creation of the account, go to your Kaggle Account page and scroll down to API section. + +

+ +

+ + - Click on Create New API Token. A new API token in the form of kaggle.json file will be created which you can save locally. The kaggle.json file contains your Kaggle username and key. + - Once you have the API credentials, run the following command in the terminal with the username and key from the kaggle.json file that you just saved. + + ``` + kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= + + ``` + This creates a secret for our credentials which can then be mounted on our pods. + + - Next create a yaml file with the following code in it. This would then be used to create a pod-default resource to mount the secret to any pod with a specific label(in our case kaggle-secret =true) + + ``` + apiVersion: "kubeflow.org/v1alpha1" + kind: PodDefault + metadata: + name: kaggle-access + spec: + selector: + matchLabels: + kaggle-secret: "true" + desc: "kaggle-access" + volumeMounts: + - name: secret-volume + mountPath: /secret/kaggle + volumes: + - name: secret-volume + secret: + secretName: kaggle-secret + + ``` + - To create a pod-default resource, run the following command, + + ``` + kubectl apply -f + + ``` + You can check out the following [link](https://support.arrikto.com/hc/en-us/articles/6335158153489-Acessing-External-System-with-User-Credentials-Kaggle-Example-) for more details about accessing external system with user credentials. +6. With the completion of 5th step, you are good to start with Vanilla KFP steps. + + + +## Vanilla KFP version + +To start building out a Kubeflow pipeline, you need to get yourself acquainted with the Kubeflow Pipelines [documentation](https://www.kubeflow.org/docs/components/pipelines/sdk/build-pipeline/) to understand what the pipelines are, its components, what goes into these components. There are different ways to build out a pipeline component as mentioned [here](https://www.kubeflow.org/docs/components/pipelines/sdk/build-pipeline/#building-pipeline-components). In the following example, we are going to use the [lightweight python functions](https://www.kubeflow.org/docs/components/pipelines/sdk/python-function-components/) based components for building up the pipeline. + +### Step 1: Install the Kubeflow Pipeline SDK and import the required kfp packages to run the pipeline + +From kfp, we will be using [func_to_container_op](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.func_to_container_op) which would help in building the factory function from the python function and we will use [InputPath](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.InputPath) and [OutputPath](https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.OutputPath) from the components package to pass the paths of the files or models to these tasks. The [passing of data](https://www.kubeflow.org/docs/components/pipelines/sdk/python-function-components/#pass-data) is being implemented by kfp’s supported data passing mechanism. InputPath and OutputPath is how you pass on the data or model between the components. For [passing values](https://www.kubeflow.org/docs/components/pipelines/sdk/python-function-components/#passing-parameters-by-value), we are using NamedTuples which allows us to send multiple values between components. + +### Step 2: Next build out the pipeline components + +Our Kubeflow pipeline is broken down into five pipeline components: + +- Download the data from Kaggle +- Load and Preprocess the data +- Creating Sparse Matrix +- Train data +- Predictions + +We convert each python function to a factory function using the func_to_container_op which will then be converted to a pipeline task for our pipeline function. + +### Step 3 : Creating pipeline function + +After building all the pipeline components, we have to define a pipeline function connecting all the pipeline components with appropriate inputs and outputs. This when run would generate the pipeline graph. + +Pipeline function: + +

+ +

+ + +### Step 4 : Running the pipeline using the kfp.client instance + +There are different ways to run the pipeline function as mentioned in the [documentation](https://www.kubeflow.org/docs/components/pipelines/sdk/build-pipeline/#compile-and-run-your-pipeline). We would run the pipeline using the Kubeflow Pipelines SDK client. + +

+ +

+ +Once all the cells are executed successfully, you should see two hyperlinks ‘Experiment details’ and ‘Run details’. Click on ‘Run details’ link to observe the pipeline running. + +The final pipeline graph would look as follow: + +

+ +

+ +## Kale KFP version + +For the Kaggle notebook example, we are using [Kubeflow as a Service](https://www.arrikto.com/kubeflow-as-a-service/). If you are using Kubeflow as a Service then Kale comes preinstalled. For users with a different Kubeflow setup, you can refer to the [GitHub link](https://github.com/kubeflow-kale/kale#getting-started) for installing the Kale JupyterLab extension on your setup. + +### Step 1: Install all the required packages + +Run the first code cell to install all the required packages (not available under the standard python library) by using the requirements.txt file. Restart the kernel after installation. + +### Step 2: Download the data from Kaggle + +Run the second code cell to download the relevant data from Kaggle using the Kaggle Public API. You will require the API credentials from the kaggle.json file you got earlier in the Initial Steps. For the Kale notebook version, you don't have to create the secret, just need the API credentials to download the data. Once the code cell is run, you should see a new "data" directory being created with the zip files downloaded and unzipped. Please ensure that you run the cell only once so you don't create nested directories. Restart the kernel before running the code cell again. + +### Step 3: Annotate the notebook with Kale tags + +The Kale notebook in the directory is already annotated. To see the annotations, open up the Kale Deployment panel and click on the Enable switch button. Once you have it switched on, you should see the following: + +

+ +

+ +Please take time to understand how each cell is annotated by clicking on the cell and checking out the tag being used and what are is its dependencies. Kale provides us with six tags for annotations: + +- Imports +- Functions +- Pipeline Parameters +- Pipeline Metrics +- Pipeline Step +- Skip Cell + +You can also see the tags being created by checking out the Cell Metadata by clicking on the Property Inspector above the Kale Deployment Panel button. + +

+ +

+ +### Step 2: Run the Kubeflow Pipeline + +Once you’ve tagged your notebook, click on the “Compile and Run” button in the Kale widget. Kale will perform the following tasks for you: + +- Validate the notebook +- Take a snapshot +- Compile the notebook +- Upload the pipeline +- Run the pipeline + +In the “Running pipeline” output, click on the “View” hyperlink. This will take you directly to the runtime execution graph where you can watch your pipeline execute and update in real-time. + +

+ +

+ +## Note: +Both notebooks have been tested out and the whole pipeline run for both the Vanilla KFP and the Kale KFP versions take around 2hrs. Most of the time is being consumed in the predictions pipeline stage. In case of any error, please test out with the following docker image. + +Notebook server docker image used: gcr.io/arrikto/jupyter-kale-py36@sha256:dd3f92ca66b46d247e4b9b6a9d84ffbb368646263c2e3909473c3b851f3fe198 + +If the error persists, please raise an issue. diff --git a/jpx-tokyo-stock-exchange-kaggle-competition/helper-files/local_api.py b/jpx-tokyo-stock-exchange-kaggle-competition/helper-files/local_api.py index 911c8d5d..abe06fa8 100644 --- a/jpx-tokyo-stock-exchange-kaggle-competition/helper-files/local_api.py +++ b/jpx-tokyo-stock-exchange-kaggle-competition/helper-files/local_api.py @@ -1,94 +1,94 @@ -import pandas as pd, os, numpy as np - -def calc_spread_return_per_day(df, portfolio_size, toprank_weight_ratio): - """ - Args: - df (pd.DataFrame): predicted results - portfolio_size (int): # of equities to buy/sell - toprank_weight_ratio (float): the relative weight of the most highly ranked stock compared to the least. - Returns: - (float): spread return - """ - assert df['Rank'].min() == 0 - assert df['Rank'].max() == len(df['Rank']) - 1 - weights = np.linspace(start=toprank_weight_ratio, stop=1, num=portfolio_size) - purchase = (df.sort_values(by='Rank')['Target'][:portfolio_size] * weights).sum() / weights.mean() - short = (df.sort_values(by='Rank', ascending=False)['Target'][:portfolio_size] * weights).sum() / weights.mean() - return purchase - short - -def calc_spread_return_sharpe(df: pd.DataFrame, portfolio_size: int = 200, toprank_weight_ratio: float = 2) -> float: - """ - Args: - df (pd.DataFrame): predicted results - portfolio_size (int): # of equities to buy/sell - toprank_weight_ratio (float): the relative weight of the most highly ranked stock compared to the least. - Returns: - (float): sharpe ratio - """ - buf = df.groupby('Date').apply(calc_spread_return_per_day, portfolio_size, toprank_weight_ratio) - sharpe_ratio = buf.mean() / buf.std() - return sharpe_ratio, buf - -class iter_test(): - def __init__(self, prices, options, financials, trades, secondary_prices, myapi): - self.myapi = myapi - self.dates = sorted(list(prices['Date'].unique())) - self.prices = prices.groupby('Date') - self.options = options.groupby('Date') - self.financials = financials.groupby('Date') - self.trades = trades.groupby('Date') - self.secondary_prices = secondary_prices.groupby('Date') - self.idx = 0 - def __next__(self): - if self.idx == len(self.dates): - self.myapi.submission = pd.concat(self.myapi.submission) - os.getcwd() - self.myapi.submission.to_csv('local_submission.csv', index=False) - self.idx = 0 - raise StopIteration - else: - prices = self.prices.get_group(self.dates[self.idx]) - options = self.options.get_group(self.dates[self.idx]) - financials = self.financials.get_group(self.dates[self.idx]) - trades = self.trades.get_group(self.dates[self.idx]) - secondary_prices = self.secondary_prices.get_group(self.dates[self.idx]) - sample_submission = pd.DataFrame(prices['Date'].copy(), columns = ['Date']) - sample_submission['SecuritiesCode'] = prices['SecuritiesCode'].copy() - self.idx += 1 - return prices, options, financials, trades, secondary_prices, sample_submission.reset_index(drop=True) - def __iter__(self): - return self - def __len__(self): - return len(self.dates) - -class local_api(): - def __init__(self, data_dir, start_date='2021-12-06', end_date='2022-02-28'): - """ - This module simulates the online API in a local environment, in order for people to estimate running time and memory. - Parameters: - data_dir: directory in which data files are stored. - start_date: str, evaluation starting date. - end_date: str, evaluation ending date. - """ - self.prices = pd.read_csv(os.path.join(data_dir, 'stock_prices.csv')) - self.options = pd.read_csv(os.path.join(data_dir, 'options.csv')) - self.financials = pd.read_csv(os.path.join(data_dir, 'financials.csv')) - self.trades = pd.read_csv(os.path.join(data_dir, 'trades.csv')) - self.secondary_prices = pd.read_csv(os.path.join(data_dir, 'secondary_stock_prices.csv')) - self.prices = self.prices.loc[(self.prices['Date'] >= start_date) & (self.prices['Date'] <= end_date)] - self.options = self.options.loc[(self.options['Date'] >= start_date) & (self.options['Date'] <= end_date)] - self.financials = self.financials.loc[(self.financials['Date'] >= start_date) & (self.financials['Date'] <= end_date)] - self.trades = self.trades.loc[(self.trades['Date'] >= start_date) & (self.trades['Date'] <= end_date)] - self.secondary_prices = self.secondary_prices.loc[(self.secondary_prices['Date'] >= start_date) &\ - (self.secondary_prices['Date'] <= end_date)] - self.gt_prices = self.prices[['Date', 'SecuritiesCode', 'Target']].copy() - self.prices.drop(['Target'], inplace=True, axis = 1) - def make_env(self): - return self - def iter_test(self): - self.submission = [] - return iter_test(self.prices, self.options, self.financials, self.trades, self.secondary_prices, self) - def predict(self, prediction): - self.submission.append(prediction) - def score(self): +import pandas as pd, os, numpy as np + +def calc_spread_return_per_day(df, portfolio_size, toprank_weight_ratio): + """ + Args: + df (pd.DataFrame): predicted results + portfolio_size (int): # of equities to buy/sell + toprank_weight_ratio (float): the relative weight of the most highly ranked stock compared to the least. + Returns: + (float): spread return + """ + assert df['Rank'].min() == 0 + assert df['Rank'].max() == len(df['Rank']) - 1 + weights = np.linspace(start=toprank_weight_ratio, stop=1, num=portfolio_size) + purchase = (df.sort_values(by='Rank')['Target'][:portfolio_size] * weights).sum() / weights.mean() + short = (df.sort_values(by='Rank', ascending=False)['Target'][:portfolio_size] * weights).sum() / weights.mean() + return purchase - short + +def calc_spread_return_sharpe(df: pd.DataFrame, portfolio_size: int = 200, toprank_weight_ratio: float = 2) -> float: + """ + Args: + df (pd.DataFrame): predicted results + portfolio_size (int): # of equities to buy/sell + toprank_weight_ratio (float): the relative weight of the most highly ranked stock compared to the least. + Returns: + (float): sharpe ratio + """ + buf = df.groupby('Date').apply(calc_spread_return_per_day, portfolio_size, toprank_weight_ratio) + sharpe_ratio = buf.mean() / buf.std() + return sharpe_ratio, buf + +class iter_test(): + def __init__(self, prices, options, financials, trades, secondary_prices, myapi): + self.myapi = myapi + self.dates = sorted(list(prices['Date'].unique())) + self.prices = prices.groupby('Date') + self.options = options.groupby('Date') + self.financials = financials.groupby('Date') + self.trades = trades.groupby('Date') + self.secondary_prices = secondary_prices.groupby('Date') + self.idx = 0 + def __next__(self): + if self.idx == len(self.dates): + self.myapi.submission = pd.concat(self.myapi.submission) + os.getcwd() + self.myapi.submission.to_csv('local_submission.csv', index=False) + self.idx = 0 + raise StopIteration + else: + prices = self.prices.get_group(self.dates[self.idx]) + options = self.options.get_group(self.dates[self.idx]) + financials = self.financials.get_group(self.dates[self.idx]) + trades = self.trades.get_group(self.dates[self.idx]) + secondary_prices = self.secondary_prices.get_group(self.dates[self.idx]) + sample_submission = pd.DataFrame(prices['Date'].copy(), columns = ['Date']) + sample_submission['SecuritiesCode'] = prices['SecuritiesCode'].copy() + self.idx += 1 + return prices, options, financials, trades, secondary_prices, sample_submission.reset_index(drop=True) + def __iter__(self): + return self + def __len__(self): + return len(self.dates) + +class local_api(): + def __init__(self, data_dir, start_date='2021-12-06', end_date='2022-02-28'): + """ + This module simulates the online API in a local environment, in order for people to estimate running time and memory. + Parameters: + data_dir: directory in which data files are stored. + start_date: str, evaluation starting date. + end_date: str, evaluation ending date. + """ + self.prices = pd.read_csv(os.path.join(data_dir, 'stock_prices.csv')) + self.options = pd.read_csv(os.path.join(data_dir, 'options.csv')) + self.financials = pd.read_csv(os.path.join(data_dir, 'financials.csv')) + self.trades = pd.read_csv(os.path.join(data_dir, 'trades.csv')) + self.secondary_prices = pd.read_csv(os.path.join(data_dir, 'secondary_stock_prices.csv')) + self.prices = self.prices.loc[(self.prices['Date'] >= start_date) & (self.prices['Date'] <= end_date)] + self.options = self.options.loc[(self.options['Date'] >= start_date) & (self.options['Date'] <= end_date)] + self.financials = self.financials.loc[(self.financials['Date'] >= start_date) & (self.financials['Date'] <= end_date)] + self.trades = self.trades.loc[(self.trades['Date'] >= start_date) & (self.trades['Date'] <= end_date)] + self.secondary_prices = self.secondary_prices.loc[(self.secondary_prices['Date'] >= start_date) &\ + (self.secondary_prices['Date'] <= end_date)] + self.gt_prices = self.prices[['Date', 'SecuritiesCode', 'Target']].copy() + self.prices.drop(['Target'], inplace=True, axis = 1) + def make_env(self): + return self + def iter_test(self): + self.submission = [] + return iter_test(self.prices, self.options, self.financials, self.trades, self.secondary_prices, self) + def predict(self, prediction): + self.submission.append(prediction) + def score(self): return calc_spread_return_sharpe(self.submission.merge(self.gt_prices))[0] \ No newline at end of file diff --git a/mnist/Makefile b/mnist/Makefile old mode 100755 new mode 100644 diff --git a/named_entity_recognition/components/build_components.sh b/named_entity_recognition/components/build_components.sh old mode 100755 new mode 100644 diff --git a/named_entity_recognition/components/copy_specification.sh b/named_entity_recognition/components/copy_specification.sh old mode 100755 new mode 100644 diff --git a/named_entity_recognition/components/deploy/build_image.sh b/named_entity_recognition/components/deploy/build_image.sh old mode 100755 new mode 100644 diff --git a/named_entity_recognition/components/preprocess/build_image.sh b/named_entity_recognition/components/preprocess/build_image.sh old mode 100755 new mode 100644 diff --git a/named_entity_recognition/components/train/build_image.sh b/named_entity_recognition/components/train/build_image.sh old mode 100755 new mode 100644 diff --git a/named_entity_recognition/routine/build_routine.sh b/named_entity_recognition/routine/build_routine.sh old mode 100755 new mode 100644 diff --git a/natural-language-processing-with-disaster-tweets-kaggle-competition/requirements.txt b/natural-language-processing-with-disaster-tweets-kaggle-competition/requirements.txt index ddca17cf..06eef536 100644 --- a/natural-language-processing-with-disaster-tweets-kaggle-competition/requirements.txt +++ b/natural-language-processing-with-disaster-tweets-kaggle-competition/requirements.txt @@ -1,7 +1,7 @@ -matplotlib==3.3.4 -seaborn==0.9.0 -nltk==3.6.7 -scikit-learn==0.23.2 -gensim==4.2.0 -tensorflow==2.3.0 -wget==3.2 +matplotlib==3.3.4 +seaborn==0.9.0 +nltk==3.6.7 +scikit-learn==0.23.2 +gensim==4.2.0 +tensorflow==2.3.0 +wget==3.2 diff --git a/object_detection/ks-app/environments/base.libsonnet b/object_detection/ks-app/environments/base.libsonnet deleted file mode 100644 index a129affb..00000000 --- a/object_detection/ks-app/environments/base.libsonnet +++ /dev/null @@ -1,4 +0,0 @@ -local components = std.extVar("__ksonnet/components"); -components + { - // Insert user-specified overrides here. -} diff --git a/openvaccine-kaggle-competition/Readme.md b/openvaccine-kaggle-competition/Readme.md index 11ccd441..35c19405 100644 --- a/openvaccine-kaggle-competition/Readme.md +++ b/openvaccine-kaggle-competition/Readme.md @@ -1,378 +1,378 @@ -# Objective - -This example is based on the Titanic OpenVaccine competition (https://www.kaggle.com/c/stanford-covid-vaccine). The objective of this exercise is to develop models and design rules for RNA degradation. - -## Environment - -This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 and ARM based system which includes all Intel and AMD based CPU's and M1/M2 series Macbooks. - -## Step 1: Setup Kubeflow as a Service - -- If you haven’t already, sign up (https://www.arrikto.com/kubeflow-as-a-service/) -- Deploy Kubeflow - -## Step 2: Launch a Notebook Server - -- Bump memory to 2GB and vCPUs to 2 - -## Step 3: Clone the Project Repo to Your Notebook - -- (Kubeflow as a Service) Open up a terminal in the Notebook Server and git clone the `kubeflow/examples` repository -``` -git clone https://github.com/kubeflow/examples -``` -## Step 4: Setup DockerHub and Docker - -- If you haven’t already, sign up (https://hub.docker.com/) for DockerHub -- If you haven’t already, install Docker Desktop locally (https://www.docker.com/products/docker-desktop/) OR install the Docker command line utility (https://docs.docker.com/get-docker/) and enter `sudo docker login` command in your terminal and log into Docker with your your DockerHub username and password - - -## Step 5: Setup Kaggle - -- If you haven’t already done so, sign up (https://www.kaggle.com/) for Kaggle -- (On Kaggle) Generate an API token (https://www.kaggle.com/docs/api) -- (Kubeflow as a Service) Create a Kubernetes secret - -``` -kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= -``` - -## Step 6: Install Git - -- (Locally) If you don’t have it already, install Git (https://github.com/git-guides/install-git) - -## Step 7: Clone the Project Repo Locally - -- (Locally) Git clone the `kubeflow/examples` repository -``` -git clone https://github.com/kubeflow/examples -``` - -## Step 8: Create a PodDefault Resource - -- (Kubeflow as a Service) Navigate to the `openvaccine-kaggle-competition` directory -- Create a `resource.yaml` file - -resource.yaml: -``` -apiVersion: "kubeflow.org/v1alpha1" -kind: PodDefault -metadata: - name: kaggle-access -spec: - selector: - matchLabels: - kaggle-secret: "true" - desc: "kaggle-access" - volumeMounts: - - name: secret-volume - mountPath: /secret/kaggle - volumes: - - name: secret-volume - secret: - secretName: kaggle-secret -``` - -![image2](https://user-images.githubusercontent.com/17012391/177001253-3e525eb6-3415-428c-a52b-11803326af6b.png) - -- Apply created resource using: `kubectl apply -f resource.yaml` - -## Step 9: Explore the `load-data` directory - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/load-data` directory -- Open up the `load.py` file -- Note the code in this file that will perform the actions required in the “load-data” pipeline step - -image7 - -## Step 10: Build the `load-data` Docker Image - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/load-data` directory -- Build the Docker image if locally you are using arm64 (Apple M1) - -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 11: Push the `load-data` Docker Image to DockerHub - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/load-data` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 12: Explore the `preprocess-data` directory - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/preprocess-data` directory -- Open up the `preprocess.py` file -- Note the code in this file that will perform the actions required in the “preprocess” pipeline step - -image5 - - -## Step 13: Explore the `preprocess-data` directory -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/preprocess-data` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 14: Push the `preprocess-data` Docker Image to DockerHub - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/preprocess-data` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 15: Explore the `model-training` directory - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-training` directory -- Open up the `model.py` file -- Note the code in this file that will perform the actions required in the “train” pipeline step - -![image4](https://user-images.githubusercontent.com/17012391/177001740-a63f190c-284e-4328-ba01-17dbdbe61cee.png) - -## Step 16: Build the `model-training` Docker Image - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-training` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 17: Push the `model-training` Docker Image to DockerHub - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-training` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 18: Explore the `model-evaluation` directory - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-evaluation` directory -- Open up the `eval.py` file -- Note the code in this file that will perform the actions required in the “test” pipeline step - -![image1](https://user-images.githubusercontent.com/17012391/177001951-1f7b13b9-adea-48c1-89a7-d8dc67214133.png) - - -## Step 19: Build the `model-evaluation` Docker Image - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-evaluation` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 20: Push the `model-evaluation` Docker Image to DockerHub - -- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-evaluation` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` -## Step 21: Modify the openvaccine-kaggle-competiton-kfp.py file - -- (Kubeflow as a Service) Navigate to the `openvaccine-kaggle-competition` directory -- Update the `openvaccine-kaggle-competiton-kfp.py` with accurate Docker Image inputs - -``` - return dsl.ContainerOp( - name = 'load-data', - image = '/:', - -—----- - -def GetMsg(comp1): - return dsl.ContainerOp( - name = 'preprocess', - image = '/:', - -—----- - -def Train(comp2, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): - return dsl.ContainerOp( - name = 'train', - image = '/:', - -—----- - -def Eval(comp1, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): - return dsl.ContainerOp( - name = 'Evaluate', - image = '/:', -``` - -## Step 22: Generate a KFP Pipeline yaml File - -- (Locally) Navigate to the `openvaccine-kaggle-competition` directory and delete the existing `openvaccine-kaggle-competition-kfp.yaml` file -- (Kubeflow as a Service) Navigate to the openvaccine-kaggle-competition directory - -Build a python virtual environment : - - -Step a) Update pip -``` -python3 -m pip install --upgrade pip -``` - -Step b) Install virtualenv -``` -sudo pip3 install virtualenv -``` - -Step c) Check the installed version of venv -``` -virtualenv --version -``` - -Step d) Name your virtual enviornment as kfp -``` -virtualenv kfp -``` - -Step e) Activate your venv. -``` -source kfp/bin/activate -``` - -After this virtual environment will get activated. Now in our activated venv we need to install following packages: -``` -sudo apt-get update -sudo apt-get upgrade -sudo apt-get install -y git python3-pip - -python3 -m pip install kfp==1.1.2 -``` - -After installing packages create the yaml file - -Inside venv point your terminal to a path which contains our kfp file to build pipeline (openvaccine-kaggle-competition-kfp.py) and run these commands to generate a `yaml` file for the Pipeline: - -``` -python3 openvaccine-kaggle-competition-kfp.py -``` -image3 - -Download the `openvaccine-kaggle-competition-kfp.yaml` file that was created to your local `openvaccine-kaggle-competition` directory - -## Step 23: Create an Experiment - -- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Experiments (KFP) > Create Experiment view -- Name the experiment and click Next -- Click on Experiments (KFP) to view the experiment you just created - -## Step 24: Create a Pipeline - - -- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Pipelines > +Upload Pipeline view -- Name the pipeline -- Click on Upload a file -- Upload the local `openvaccine-kaggle-competition-kfp.yaml` file -- Click Create - -## Step 25: Create a Run - -- (Kubeflow as a Service) Click on Create Run in the view from the previous step -- Choose the experiment we created in Step 23 -- Input your desired run parameters. For example: -``` -TRIAL = 1 -EPOCHS = 2 -BATCH_SIZE = 64 -EMBED_DIM = 100 -HIDDEN_DIM = 128 -DROPOUT = .2 -SP_DROPOUT = .3 -TRAIN_SEQUENCE_LENGTH = 107 -``` -- Click Start -- Click on the run name to view the runtime execution graph - - - - -![image6](https://user-images.githubusercontent.com/17012391/177002214-8258e3fa-e669-43cc-979d-70c1059f6aae.png) - - - - - - - - - - - - - - - - - - - - -## Troubleshooting Tips: -While running the pipeline as mentioned above you may come across this error: -![kaggle-secret-error-01](https://user-images.githubusercontent.com/17012391/175290593-aac58d80-0d9f-47bd-bd20-46e6f5207210.PNG) - -errorlog: - -``` -kaggle.rest.ApiException: (403) -Reason: Forbidden -HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Thu, 23 Jun 2022 11:31:18 GMT', 'Access-Control-Allow-Credentials': 'true', 'Set-Cookie': 'ka_sessionid=6817a347c75399a531148e19cad0aaeb; max-age=2626560; path=/, GCLB=CIGths3--ebbUg; path=/; HttpOnly', 'Transfer-Encoding': 'chunked', 'Vary': -HTTP response body: b'{"code":403,"message":"You must accept this competition\\u0027s rules before you\\u0027ll be able to download files."}' - -``` -This error occours for two reasons: -- Your Kaggle account is not verified with your phone number. -- Rules for this specific competitions are not accepted. - -Lets accept Rules of competition -![rules](https://user-images.githubusercontent.com/17012391/175306310-10808262-07ce-4952-8fb0-3b7754e9fb46.png) - -Click on "I Understand and Accept". After this you will be prompted to verify your account using your phone number: -![kaggle-secret-error-03](https://user-images.githubusercontent.com/17012391/175291608-daad1a47-119a-4e47-b48b-4f878d65ddd7.PNG) - -Add your phone number and Kaggle will send the code to your number, enter this code and verify your account. ( Note: pipeline wont run if your Kaggle account is not verified ) - -## Success -After the kaggle account is verified pipeline run is successful we will get the following: -Screenshot 2022-06-06 at 3 00 51 PM +# Objective + +This example is based on the Titanic OpenVaccine competition (https://www.kaggle.com/c/stanford-covid-vaccine). The objective of this exercise is to develop models and design rules for RNA degradation. + +## Environment + +This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 and ARM based system which includes all Intel and AMD based CPU's and M1/M2 series Macbooks. + +## Step 1: Setup Kubeflow as a Service + +- If you haven’t already, sign up (https://www.arrikto.com/kubeflow-as-a-service/) +- Deploy Kubeflow + +## Step 2: Launch a Notebook Server + +- Bump memory to 2GB and vCPUs to 2 + +## Step 3: Clone the Project Repo to Your Notebook + +- (Kubeflow as a Service) Open up a terminal in the Notebook Server and git clone the `kubeflow/examples` repository +``` +git clone https://github.com/kubeflow/examples +``` +## Step 4: Setup DockerHub and Docker + +- If you haven’t already, sign up (https://hub.docker.com/) for DockerHub +- If you haven’t already, install Docker Desktop locally (https://www.docker.com/products/docker-desktop/) OR install the Docker command line utility (https://docs.docker.com/get-docker/) and enter `sudo docker login` command in your terminal and log into Docker with your your DockerHub username and password + + +## Step 5: Setup Kaggle + +- If you haven’t already done so, sign up (https://www.kaggle.com/) for Kaggle +- (On Kaggle) Generate an API token (https://www.kaggle.com/docs/api) +- (Kubeflow as a Service) Create a Kubernetes secret + +``` +kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= +``` + +## Step 6: Install Git + +- (Locally) If you don’t have it already, install Git (https://github.com/git-guides/install-git) + +## Step 7: Clone the Project Repo Locally + +- (Locally) Git clone the `kubeflow/examples` repository +``` +git clone https://github.com/kubeflow/examples +``` + +## Step 8: Create a PodDefault Resource + +- (Kubeflow as a Service) Navigate to the `openvaccine-kaggle-competition` directory +- Create a `resource.yaml` file + +resource.yaml: +``` +apiVersion: "kubeflow.org/v1alpha1" +kind: PodDefault +metadata: + name: kaggle-access +spec: + selector: + matchLabels: + kaggle-secret: "true" + desc: "kaggle-access" + volumeMounts: + - name: secret-volume + mountPath: /secret/kaggle + volumes: + - name: secret-volume + secret: + secretName: kaggle-secret +``` + +![image2](https://user-images.githubusercontent.com/17012391/177001253-3e525eb6-3415-428c-a52b-11803326af6b.png) + +- Apply created resource using: `kubectl apply -f resource.yaml` + +## Step 9: Explore the `load-data` directory + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/load-data` directory +- Open up the `load.py` file +- Note the code in this file that will perform the actions required in the “load-data” pipeline step + +image7 + +## Step 10: Build the `load-data` Docker Image + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/load-data` directory +- Build the Docker image if locally you are using arm64 (Apple M1) + +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 11: Push the `load-data` Docker Image to DockerHub + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/load-data` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 12: Explore the `preprocess-data` directory + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/preprocess-data` directory +- Open up the `preprocess.py` file +- Note the code in this file that will perform the actions required in the “preprocess” pipeline step + +image5 + + +## Step 13: Explore the `preprocess-data` directory +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/preprocess-data` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 14: Push the `preprocess-data` Docker Image to DockerHub + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/preprocess-data` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 15: Explore the `model-training` directory + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-training` directory +- Open up the `model.py` file +- Note the code in this file that will perform the actions required in the “train” pipeline step + +![image4](https://user-images.githubusercontent.com/17012391/177001740-a63f190c-284e-4328-ba01-17dbdbe61cee.png) + +## Step 16: Build the `model-training` Docker Image + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-training` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 17: Push the `model-training` Docker Image to DockerHub + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-training` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 18: Explore the `model-evaluation` directory + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-evaluation` directory +- Open up the `eval.py` file +- Note the code in this file that will perform the actions required in the “test” pipeline step + +![image1](https://user-images.githubusercontent.com/17012391/177001951-1f7b13b9-adea-48c1-89a7-d8dc67214133.png) + + +## Step 19: Build the `model-evaluation` Docker Image + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-evaluation` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 20: Push the `model-evaluation` Docker Image to DockerHub + +- (Locally) Navigate to the `openvaccine-kaggle-competition/pipeline-components/model-evaluation` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` +## Step 21: Modify the openvaccine-kaggle-competiton-kfp.py file + +- (Kubeflow as a Service) Navigate to the `openvaccine-kaggle-competition` directory +- Update the `openvaccine-kaggle-competiton-kfp.py` with accurate Docker Image inputs + +``` + return dsl.ContainerOp( + name = 'load-data', + image = '/:', + +—----- + +def GetMsg(comp1): + return dsl.ContainerOp( + name = 'preprocess', + image = '/:', + +—----- + +def Train(comp2, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): + return dsl.ContainerOp( + name = 'train', + image = '/:', + +—----- + +def Eval(comp1, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): + return dsl.ContainerOp( + name = 'Evaluate', + image = '/:', +``` + +## Step 22: Generate a KFP Pipeline yaml File + +- (Locally) Navigate to the `openvaccine-kaggle-competition` directory and delete the existing `openvaccine-kaggle-competition-kfp.yaml` file +- (Kubeflow as a Service) Navigate to the openvaccine-kaggle-competition directory + +Build a python virtual environment : + + +Step a) Update pip +``` +python3 -m pip install --upgrade pip +``` + +Step b) Install virtualenv +``` +sudo pip3 install virtualenv +``` + +Step c) Check the installed version of venv +``` +virtualenv --version +``` + +Step d) Name your virtual enviornment as kfp +``` +virtualenv kfp +``` + +Step e) Activate your venv. +``` +source kfp/bin/activate +``` + +After this virtual environment will get activated. Now in our activated venv we need to install following packages: +``` +sudo apt-get update +sudo apt-get upgrade +sudo apt-get install -y git python3-pip + +python3 -m pip install kfp==1.1.2 +``` + +After installing packages create the yaml file + +Inside venv point your terminal to a path which contains our kfp file to build pipeline (openvaccine-kaggle-competition-kfp.py) and run these commands to generate a `yaml` file for the Pipeline: + +``` +python3 openvaccine-kaggle-competition-kfp.py +``` +image3 + +Download the `openvaccine-kaggle-competition-kfp.yaml` file that was created to your local `openvaccine-kaggle-competition` directory + +## Step 23: Create an Experiment + +- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Experiments (KFP) > Create Experiment view +- Name the experiment and click Next +- Click on Experiments (KFP) to view the experiment you just created + +## Step 24: Create a Pipeline + + +- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Pipelines > +Upload Pipeline view +- Name the pipeline +- Click on Upload a file +- Upload the local `openvaccine-kaggle-competition-kfp.yaml` file +- Click Create + +## Step 25: Create a Run + +- (Kubeflow as a Service) Click on Create Run in the view from the previous step +- Choose the experiment we created in Step 23 +- Input your desired run parameters. For example: +``` +TRIAL = 1 +EPOCHS = 2 +BATCH_SIZE = 64 +EMBED_DIM = 100 +HIDDEN_DIM = 128 +DROPOUT = .2 +SP_DROPOUT = .3 +TRAIN_SEQUENCE_LENGTH = 107 +``` +- Click Start +- Click on the run name to view the runtime execution graph + + + + +![image6](https://user-images.githubusercontent.com/17012391/177002214-8258e3fa-e669-43cc-979d-70c1059f6aae.png) + + + + + + + + + + + + + + + + + + + + +## Troubleshooting Tips: +While running the pipeline as mentioned above you may come across this error: +![kaggle-secret-error-01](https://user-images.githubusercontent.com/17012391/175290593-aac58d80-0d9f-47bd-bd20-46e6f5207210.PNG) + +errorlog: + +``` +kaggle.rest.ApiException: (403) +Reason: Forbidden +HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Thu, 23 Jun 2022 11:31:18 GMT', 'Access-Control-Allow-Credentials': 'true', 'Set-Cookie': 'ka_sessionid=6817a347c75399a531148e19cad0aaeb; max-age=2626560; path=/, GCLB=CIGths3--ebbUg; path=/; HttpOnly', 'Transfer-Encoding': 'chunked', 'Vary': +HTTP response body: b'{"code":403,"message":"You must accept this competition\\u0027s rules before you\\u0027ll be able to download files."}' + +``` +This error occours for two reasons: +- Your Kaggle account is not verified with your phone number. +- Rules for this specific competitions are not accepted. + +Lets accept Rules of competition +![rules](https://user-images.githubusercontent.com/17012391/175306310-10808262-07ce-4952-8fb0-3b7754e9fb46.png) + +Click on "I Understand and Accept". After this you will be prompted to verify your account using your phone number: +![kaggle-secret-error-03](https://user-images.githubusercontent.com/17012391/175291608-daad1a47-119a-4e47-b48b-4f878d65ddd7.PNG) + +Add your phone number and Kaggle will send the code to your number, enter this code and verify your account. ( Note: pipeline wont run if your Kaggle account is not verified ) + +## Success +After the kaggle account is verified pipeline run is successful we will get the following: +Screenshot 2022-06-06 at 3 00 51 PM diff --git a/openvaccine-kaggle-competition/openvaccine-kaggle-competition-kfp.py b/openvaccine-kaggle-competition/openvaccine-kaggle-competition-kfp.py index 120859e2..a79f9f71 100644 --- a/openvaccine-kaggle-competition/openvaccine-kaggle-competition-kfp.py +++ b/openvaccine-kaggle-competition/openvaccine-kaggle-competition-kfp.py @@ -1,81 +1,81 @@ -import kfp -from kfp import dsl - -def SendMsg(): - vop = dsl.VolumeOp(name="pvc", - resource_name="pvc", size='1Gi', - modes=dsl.VOLUME_MODE_RWO) - - return dsl.ContainerOp( - name = 'load-data', - image = 'hubdocker76/openvaccine:v10', - command = ['python3', 'load.py'], - - pvolumes={ - '/data': vop.volume - } - ) - -def GetMsg(comp1): - return dsl.ContainerOp( - name = 'preprocess', - image = 'hubdocker76/preprocess-data:v10', - pvolumes={ - '/data': comp1.pvolumes['/data'] - }, - command = ['python3', 'preprocess.py'] - ) - -def Train(comp2, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): - return dsl.ContainerOp( - name = 'train', - image = 'hubdocker76/model-training:v21', - command = ['python3', 'model.py'], - arguments=[ - '--LR', trial, - '--EPOCHS', epoch, - '--BATCH_SIZE', batchsize, - '--EMBED_DIM', embeddim, - '--HIDDEN_DIM', hiddendim, - '--DROPOUT', dropout, - '--SP_DROPOUT', spdropout, - '--TRAIN_SEQUENCE_LENGTH', trainsequencelength - ], - pvolumes={ - '/data': comp2.pvolumes['/data'] - } - ) - -def Eval(comp1, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): - return dsl.ContainerOp( - name = 'Evaluate', - image = 'hubdocker76/eval:v4', - arguments=[ - '--LR', trial, - '--EPOCHS', epoch, - '--BATCH_SIZE', batchsize, - '--EMBED_DIM', embeddim, - '--HIDDEN_DIM', hiddendim, - '--DROPOUT', dropout, - '--SP_DROPOUT', spdropout, - '--TRAIN_SEQUENCE_LENGTH', trainsequencelength - ], - pvolumes={ - '/data': comp1.pvolumes['/data'] - }, - command = ['python3', 'eval.py'] - ) - -@dsl.pipeline( - name = 'openvaccine', - description = 'pipeline to run openvaccine') - -def passing_parameter(trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): - comp1 = SendMsg().add_pod_label("kaggle-secret", "true") - comp2 = GetMsg(comp1) - comp3 = Train(comp2, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength) - comp4 = Eval(comp3, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength) - -if __name__ == '__main__': - import kfp.compiler as compiler - compiler.Compiler().compile(passing_parameter, __file__[:-3]+ '.yaml') +import kfp +from kfp import dsl + +def SendMsg(): + vop = dsl.VolumeOp(name="pvc", + resource_name="pvc", size='1Gi', + modes=dsl.VOLUME_MODE_RWO) + + return dsl.ContainerOp( + name = 'load-data', + image = 'hubdocker76/openvaccine:v10', + command = ['python3', 'load.py'], + + pvolumes={ + '/data': vop.volume + } + ) + +def GetMsg(comp1): + return dsl.ContainerOp( + name = 'preprocess', + image = 'hubdocker76/preprocess-data:v10', + pvolumes={ + '/data': comp1.pvolumes['/data'] + }, + command = ['python3', 'preprocess.py'] + ) + +def Train(comp2, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): + return dsl.ContainerOp( + name = 'train', + image = 'hubdocker76/model-training:v21', + command = ['python3', 'model.py'], + arguments=[ + '--LR', trial, + '--EPOCHS', epoch, + '--BATCH_SIZE', batchsize, + '--EMBED_DIM', embeddim, + '--HIDDEN_DIM', hiddendim, + '--DROPOUT', dropout, + '--SP_DROPOUT', spdropout, + '--TRAIN_SEQUENCE_LENGTH', trainsequencelength + ], + pvolumes={ + '/data': comp2.pvolumes['/data'] + } + ) + +def Eval(comp1, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): + return dsl.ContainerOp( + name = 'Evaluate', + image = 'hubdocker76/eval:v4', + arguments=[ + '--LR', trial, + '--EPOCHS', epoch, + '--BATCH_SIZE', batchsize, + '--EMBED_DIM', embeddim, + '--HIDDEN_DIM', hiddendim, + '--DROPOUT', dropout, + '--SP_DROPOUT', spdropout, + '--TRAIN_SEQUENCE_LENGTH', trainsequencelength + ], + pvolumes={ + '/data': comp1.pvolumes['/data'] + }, + command = ['python3', 'eval.py'] + ) + +@dsl.pipeline( + name = 'openvaccine', + description = 'pipeline to run openvaccine') + +def passing_parameter(trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength): + comp1 = SendMsg().add_pod_label("kaggle-secret", "true") + comp2 = GetMsg(comp1) + comp3 = Train(comp2, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength) + comp4 = Eval(comp3, trial, epoch, batchsize, embeddim, hiddendim, dropout, spdropout, trainsequencelength) + +if __name__ == '__main__': + import kfp.compiler as compiler + compiler.Compiler().compile(passing_parameter, __file__[:-3]+ '.yaml') diff --git a/pipelines-demo/volume/volume_example.py b/pipelines-demo/volume/volume_example.py index 59ac2f1a..114868bc 100644 --- a/pipelines-demo/volume/volume_example.py +++ b/pipelines-demo/volume/volume_example.py @@ -1,63 +1,63 @@ -# Copyright 2023 kbthu. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import kfp -from kfp import dsl - -def create_pv(): - return dsl.VolumeOp( - name="create_pv", - resource_name="kfp-pvc", - size="1Gi", - modes=dsl.VOLUME_MODE_RWO - ) - - -def generate_data(vol_name: str): - cop = dsl.ContainerOp( - name='generate_data', - image='bash:5.1', - command=['sh', '-c'], - arguments=['echo $(( $RANDOM % 10 + 1 )) | tee /mnt/out.txt'] - ) - cop.container.set_image_pull_policy('IfNotPresent') - cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) - return cop - - -def use_pre_data(vol_name: str): - cop = dsl.ContainerOp( - name='use_pre_data', - image='bash:5.1', - command=['sh', '-c'], - arguments=['tail /mnt/out.txt'] - ) - cop.container.set_image_pull_policy('IfNotPresent') - cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) - return cop - - -@dsl.pipeline( - name="Kubeflow volume example", - description="Demonstrate the use case of volume on Kubeflow pipeline." -) -def volume_example(): - vop = create_pv() - cop = generate_data(vop.outputs["name"]).after(vop) - use_pre_data(vop.outputs["name"]).after(vop,cop) - - -if __name__ == "__main__": - import kfp.compiler as compiler +# Copyright 2023 kbthu. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import kfp +from kfp import dsl + +def create_pv(): + return dsl.VolumeOp( + name="create_pv", + resource_name="kfp-pvc", + size="1Gi", + modes=dsl.VOLUME_MODE_RWO + ) + + +def generate_data(vol_name: str): + cop = dsl.ContainerOp( + name='generate_data', + image='bash:5.1', + command=['sh', '-c'], + arguments=['echo $(( $RANDOM % 10 + 1 )) | tee /mnt/out.txt'] + ) + cop.container.set_image_pull_policy('IfNotPresent') + cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) + return cop + + +def use_pre_data(vol_name: str): + cop = dsl.ContainerOp( + name='use_pre_data', + image='bash:5.1', + command=['sh', '-c'], + arguments=['tail /mnt/out.txt'] + ) + cop.container.set_image_pull_policy('IfNotPresent') + cop.add_pvolumes({'/mnt': dsl.PipelineVolume(pvc=vol_name)}) + return cop + + +@dsl.pipeline( + name="Kubeflow volume example", + description="Demonstrate the use case of volume on Kubeflow pipeline." +) +def volume_example(): + vop = create_pv() + cop = generate_data(vop.outputs["name"]).after(vop) + use_pre_data(vop.outputs["name"]).after(vop,cop) + + +if __name__ == "__main__": + import kfp.compiler as compiler compiler.Compiler().compile(volume_example, __file__ + ".yaml") \ No newline at end of file diff --git a/pipelines/azurepipeline/code/deploy/Dockerfile b/pipelines/azurepipeline/code/deploy/Dockerfile old mode 100755 new mode 100644 diff --git a/pipelines/azurepipeline/code/deploy/score.py b/pipelines/azurepipeline/code/deploy/score.py old mode 100755 new mode 100644 diff --git a/pipelines/azurepipeline/code/preprocess/Dockerfile b/pipelines/azurepipeline/code/preprocess/Dockerfile old mode 100755 new mode 100644 diff --git a/pipelines/azurepipeline/code/preprocess/data.py b/pipelines/azurepipeline/code/preprocess/data.py old mode 100755 new mode 100644 diff --git a/pipelines/azurepipeline/code/register/Dockerfile b/pipelines/azurepipeline/code/register/Dockerfile old mode 100755 new mode 100644 diff --git a/pipelines/azurepipeline/code/register/register.py b/pipelines/azurepipeline/code/register/register.py old mode 100755 new mode 100644 diff --git a/pipelines/azurepipeline/code/training/Dockerfile b/pipelines/azurepipeline/code/training/Dockerfile old mode 100755 new mode 100644 diff --git a/pipelines/azurepipeline/code/training/train.py b/pipelines/azurepipeline/code/training/train.py old mode 100755 new mode 100644 diff --git a/pipelines/mnist-pipelines/deploy-service/src/deploy.sh b/pipelines/mnist-pipelines/deploy-service/src/deploy.sh old mode 100755 new mode 100644 diff --git a/pipelines/simple-notebook-pipeline/README.md b/pipelines/simple-notebook-pipeline/README.md index 3174b84a..6e562e1a 100644 --- a/pipelines/simple-notebook-pipeline/README.md +++ b/pipelines/simple-notebook-pipeline/README.md @@ -1,10 +1,10 @@ -# Simple Notebook Pipeline on GCP -This notebook shows how to compile and run a simple Kubeflow pipeline using Jupyter notebooks and Google Cloud Storage. The pipeline is very simple, and is a helpful starting point for people new to Kubeflow. - -## Setup - -### Setup notebook server -This pipeline requires you to [setup a notebook server](https://www.kubeflow.org/docs/components/notebooks/setup/) in the Kubeflow UI. After you are setup, upload this notebook and then run it in the notebook server. - -### Upload the notebook to the Kubeflow UI -In order to run this pipeline, make sure to upload the notebook to your notebook server in the Kubeflow UI. You can clone this repo in the Jupyter notebook server by connecting to the notebook server and then selecting New > Terminal. In the terminal type `git clone https://github.com/kubeflow/examples.git`. +# Simple Notebook Pipeline on GCP +This notebook shows how to compile and run a simple Kubeflow pipeline using Jupyter notebooks and Google Cloud Storage. The pipeline is very simple, and is a helpful starting point for people new to Kubeflow. + +## Setup + +### Setup notebook server +This pipeline requires you to [setup a notebook server](https://www.kubeflow.org/docs/components/notebooks/setup/) in the Kubeflow UI. After you are setup, upload this notebook and then run it in the notebook server. + +### Upload the notebook to the Kubeflow UI +In order to run this pipeline, make sure to upload the notebook to your notebook server in the Kubeflow UI. You can clone this repo in the Jupyter notebook server by connecting to the notebook server and then selecting New > Terminal. In the terminal type `git clone https://github.com/kubeflow/examples.git`. diff --git a/pytorch_mnist/Makefile b/pytorch_mnist/Makefile old mode 100755 new mode 100644 diff --git a/pytorch_mnist/serving/seldon-wrapper/build_image.sh b/pytorch_mnist/serving/seldon-wrapper/build_image.sh old mode 100755 new mode 100644 diff --git a/pytorch_mnist/training/ddp/mnist/Dockerfile.traingpu b/pytorch_mnist/training/ddp/mnist/Dockerfile.traingpu old mode 100755 new mode 100644 diff --git a/pytorch_mnist/training/ddp/mnist/build_image.sh b/pytorch_mnist/training/ddp/mnist/build_image.sh old mode 100755 new mode 100644 diff --git a/pytorch_mnist/training/ddp/mnist/mnist_DDP.py b/pytorch_mnist/training/ddp/mnist/mnist_DDP.py old mode 100755 new mode 100644 diff --git a/pytorch_mnist/web-ui/build_image.sh b/pytorch_mnist/web-ui/build_image.sh old mode 100755 new mode 100644 diff --git a/telco-customer-churn-kaggle-competition/images/... b/telco-customer-churn-kaggle-competition/images/... deleted file mode 100644 index 8b137891..00000000 --- a/telco-customer-churn-kaggle-competition/images/... +++ /dev/null @@ -1 +0,0 @@ - diff --git a/telco-customer-churn-kaggle-competition/requirements.txt b/telco-customer-churn-kaggle-competition/requirements.txt index c1db402e..b7c9aa30 100644 --- a/telco-customer-churn-kaggle-competition/requirements.txt +++ b/telco-customer-churn-kaggle-competition/requirements.txt @@ -1,6 +1,6 @@ -pandas -seaborn -lightgbm -catboost -xgboost -wget +pandas +seaborn +lightgbm +catboost +xgboost +wget diff --git a/tensorflow_cuj/text_classification/distributed_text_classification_rnn.py b/tensorflow_cuj/text_classification/distributed_text_classification_rnn.py old mode 100755 new mode 100644 diff --git a/test/copy_secret.sh b/test/copy_secret.sh old mode 100755 new mode 100644 diff --git a/titanic-kaggle-competition/Readme.md b/titanic-kaggle-competition/Readme.md index 5176c0bd..7137292e 100644 --- a/titanic-kaggle-competition/Readme.md +++ b/titanic-kaggle-competition/Readme.md @@ -1,464 +1,464 @@ -# Objective - -This example is based on the Titanic Kaggle competition (https://www.kaggle.com/c/titanic). The objective of this exercise is to use machine learning to create a model that predicts which passengers survived the Titanic shipwreck. - -## Environment - -This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 and ARM based system which includes all Intel and AMD based CPU's and M1/M2 series Macbooks - -## Step 1: Setup Kubeflow as a Service - -- If you haven’t already, sign up (https://www.arrikto.com/kubeflow-as-a-service/) -- Deploy Kubeflow - -## Step 2: Launch a Notebook Server - -- Default should work - -## Step 3: Clone the Project Repo to Your Notebook - -- (Kubeflow as a Service) Open up a terminal in the Notebook Server and git clone the `kubeflow/examples` repository -``` -git clone https://github.com/kubeflow/examples -``` - -## Step 4: Setup DockerHub and Docker - -- If you haven’t already, sign up (https://hub.docker.com/) for DockerHub -- If you haven’t already, install Docker Desktop (https://www.docker.com/products/docker-desktop/) locally OR install the Docker command line utility (https://docs.docker.com/get-docker/) and enter `sudo docker login` command in your terminal and log into Docker with your your DockerHub username and password - -## Step 5: Setup Kaggle - -- If you haven’t already done so, sign up (https://www.kaggle.com/) for Kaggle -- (On Kaggle) Generate an API token (https://www.kaggle.com/docs/api) -- (Kubeflow as a Service) Create a Kubernetes secret -``` -kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= -``` - -## Step 6: Install Git - -- (Locally) If you don’t have it already, install Git - -## Step 7: Clone the Project Repo Locally - -- (Locally) Git clone the kubeflow/examples repository -``` -git clone https://github.com/kubeflow/examples -``` -## Step 8: Create a `PodDefault` Resource - -- (Kubeflow as a Service) Navigate to the `titanic-kaggle-competition` directory -- Create a `resource.yaml` file - -resource.yaml: -``` -apiVersion: "kubeflow.org/v1alpha1" -kind: PodDefault -metadata: - name: kaggle-access -spec: - selector: - matchLabels: - kaggle-secret: "true" - desc: "kaggle-access" - volumeMounts: - - name: secret-volume - mountPath: /secret/kaggle - volumes: - - name: secret-volume - secret: - secretName: kaggle-secret -``` -Screenshot 2022-07-04 at 4 56 41 PM - -- Apply the resource.yaml file: `kubectl apply -f resource.yaml` - -## Step 9: Explore the pre-process directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/pre-process` directory -- Open up the `preprocess.py` file -- Note the code in this file that will perform the actions required in the “preprocess-data” pipeline step - -Screenshot 2022-07-04 at 5 00 01 PM - -## Step 10: Build the preprocess-data Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/pre-process` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` -## Step 11: Push the preprocess-data Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/load-data` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` -## Step 12: Explore the featureengineering directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/featureengineering` directory -- Open up the `featureengg.py` file -- Note the code in this file that will perform the actions required in the “featureengineering” pipeline step - -Screenshot 2022-07-04 at 5 02 50 PM - -## Step 13: Build the featureengineering Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/featureengineering` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` -## Step 14: Push the featureengineering Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/featureengineering` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 15: Explore the decisiontree directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/decisiontree` directory -- Open up the `decisiontree.py` file -- Note the code in this file that will perform the actions required in the “decision-tree” pipeline step - -Screenshot 2022-07-04 at 5 05 43 PM - -## Step 16: Build the decisiontree Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/decisiontree` directory -Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 17: Push the decisiontree Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/decisiontree` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` -## Step 18: Explore the logisticregression directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/logisticregression` directory -- Open up the `regression.py` file -- Note the code in this file that will perform the actions required in the “regression” pipeline step - -Screenshot 2022-07-04 at 5 08 11 PM - -## Step 19: Build the regression Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/logisticregression` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` -## Step 20: Push the regression Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/logisticregression` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` -## Step 21: Explore the naivebayes directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/naivebayes` directory -- Open up the `naivebayes.py` file -Note the code in this file that will perform the actions required in the “bayes” pipeline step -Screenshot 2022-07-04 at 5 10 36 PM - -## Step 22: Build the naivebayes Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/naivebayes` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` -## Step 23: Push the naivebayes Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/naivebayes` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` -## Step 24: Explore the randomforest directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/randomforest` directory -- Open up the `randomforest.py` file -- Note the code in this file that will perform the actions required in the “random-forest” pipeline step - - -Screenshot 2022-07-04 at 5 12 54 PM - -## Step 25: Build the random-forest Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/randomforest` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 26: Push the random-forest Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/randomforest` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` -## Step 27: Explore the svm directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/svm` directory -- Open up the `svm.py` file -- Note the code in this file that will perform the actions required in the “svm” pipeline step - -Screenshot 2022-07-04 at 5 15 23 PM - -## Step 28: Build the svm Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/svm` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` -## Step 29: Push the svm Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/svm` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` -## Step 30: Explore the results directory - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/results` directory -- Open up the `result.py` file -- Note the code in this file that will perform the actions required in the “results” pipeline step - -Screenshot 2022-07-04 at 5 18 34 PM - -## Step 31: Build the results Docker Image - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/results` directory -- Build the Docker image if locally you are using arm64 (Apple M1) -``` -docker build --platform=linux/amd64 -t /:-amd64 . -``` -- OR build the Docker image if locally you are using amd64 -``` -docker build -t /: . -``` - -## Step 32: Push the results Docker Image to DockerHub - -- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/results` directory -- Push the Docker image if locally you are using arm64 (Apple M1) -``` -docker push /:-amd64 -``` -- OR build the Docker image if locally you are using amd64 -``` -docker push /: -``` - -## Step 33: Modify the titanic-kfp.py file - -- (Kubeflow as a Service) Navigate to the `titanic-kaggle-competition` directory -- Update the `titanic-kfp.py` with accurate Docker Image inputs -``` - return dsl.ContainerOp( - name = 'Preprocess Data', - image = '/:', - -—----- - - return dsl.ContainerOp( - name='featureengineering', - image = '/:', - -—----- - - return dsl.ContainerOp( - name='regression', - image = '/:', - -—----- - - return dsl.ContainerOp( - name='bayes', - image = '/:', - -—----- - - return dsl.ContainerOp( - name='random_forest', - image = '/:', - -—----- - - return dsl.ContainerOp( - name='decision_tree', - image = '/:', - -—----- - - return dsl.ContainerOp( - name='svm', - image = '/:', - -—----- - - return dsl.ContainerOp( - name='results', - image = '/:', -``` - -## Step 34: Generate a KFP Pipeline yaml File - -- (Kubeflow as a Service) Navigate to the `titanic-kaggle-competition` directory -Build a python virtual environment: - -Step a) Update pip -``` -python3 -m pip install --upgrade pip -``` - -Step b) Install virtualenv -``` -sudo pip3 install virtualenv -``` - -Step c) Check the installed version of venv -``` -virtualenv --version -``` - -Step d) Name your virtual enviornment as kfp -``` -virtualenv kfp -``` - -Step e) Activate your venv. -``` -source kfp/bin/activate -``` - -After this virtual environment will get activated. Now in our activated venv we need to install following packages: -``` -sudo apt-get update -sudo apt-get upgrade -sudo apt-get install -y git python3-pip - -python3 -m pip install kfp==1.1.2 -``` - -After installing packages create the yaml file -``` -python3 titanic-kaggle-competition-kfp.py -``` - -Screenshot 2022-07-04 at 5 27 37 PM - - -Download the `titanic-kaggle-competition-kfp.yaml` file that was created to your local `titanic-kaggle-competition` directory. - -## Step 35: Create an Experiment - -- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Experiments (KFP) > Create Experiment view -- Name the experiment and click Next -- Click on Experiments (KFP) to view the experiment you just created - -## Step 36: Create a Pipeline - -- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Pipelines > +Upload Pipeline view -- Name the pipeline -- Click on Upload a file -- Upload the local `titanic-kaggle-competition-kfp.yaml` file -- Click Create - - -## Step 37: Create a Run - -- (Kubeflow as a Service) Click on Create Run in the view from the previous step -- Choose the experiment we created in Step 35 -- Click Start -- Click on the run name to view the runtime execution graph - - -![image10](https://user-images.githubusercontent.com/17012391/177150882-3c8abf80-2d6e-4467-9b11-7824d3909e35.png) - - -## Troubleshooting Tips: -While running the pipeline as mentioned above you may come across this error: -errorlog: - -``` -kaggle.rest.ApiException: (403) -Reason: Forbidden -HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Thu, 23 Jun 2022 11:31:18 GMT', 'Access-Control-Allow-Credentials': 'true', 'Set-Cookie': 'ka_sessionid=6817a347c75399a531148e19cad0aaeb; max-age=2626560; path=/, GCLB=CIGths3--ebbUg; path=/; HttpOnly', 'Transfer-Encoding': 'chunked', 'Vary': -HTTP response body: b'{"code":403,"message":"You must accept this competition\\u0027s rules before you\\u0027ll be able to download files."}' - -``` -This error occours for two reasons: -- Your Kaggle account is not verified with your phone number. -- Rules for this specific competitions are not accepted. - -A solution to this is please verify your Kaggle account using your phone number and accept the rules for this specific competition, untill these two steps are satisfied pipeline wont accquire data from Kaggle API and it wont run. +# Objective + +This example is based on the Titanic Kaggle competition (https://www.kaggle.com/c/titanic). The objective of this exercise is to use machine learning to create a model that predicts which passengers survived the Titanic shipwreck. + +## Environment + +This pipeline was tested using Kubeflow 1.4 and kfp 1.1.2 and x86-64 and ARM based system which includes all Intel and AMD based CPU's and M1/M2 series Macbooks + +## Step 1: Setup Kubeflow as a Service + +- If you haven’t already, sign up (https://www.arrikto.com/kubeflow-as-a-service/) +- Deploy Kubeflow + +## Step 2: Launch a Notebook Server + +- Default should work + +## Step 3: Clone the Project Repo to Your Notebook + +- (Kubeflow as a Service) Open up a terminal in the Notebook Server and git clone the `kubeflow/examples` repository +``` +git clone https://github.com/kubeflow/examples +``` + +## Step 4: Setup DockerHub and Docker + +- If you haven’t already, sign up (https://hub.docker.com/) for DockerHub +- If you haven’t already, install Docker Desktop (https://www.docker.com/products/docker-desktop/) locally OR install the Docker command line utility (https://docs.docker.com/get-docker/) and enter `sudo docker login` command in your terminal and log into Docker with your your DockerHub username and password + +## Step 5: Setup Kaggle + +- If you haven’t already done so, sign up (https://www.kaggle.com/) for Kaggle +- (On Kaggle) Generate an API token (https://www.kaggle.com/docs/api) +- (Kubeflow as a Service) Create a Kubernetes secret +``` +kubectl create secret generic kaggle-secret --from-literal=KAGGLE_USERNAME= --from-literal=KAGGLE_KEY= +``` + +## Step 6: Install Git + +- (Locally) If you don’t have it already, install Git + +## Step 7: Clone the Project Repo Locally + +- (Locally) Git clone the kubeflow/examples repository +``` +git clone https://github.com/kubeflow/examples +``` +## Step 8: Create a `PodDefault` Resource + +- (Kubeflow as a Service) Navigate to the `titanic-kaggle-competition` directory +- Create a `resource.yaml` file + +resource.yaml: +``` +apiVersion: "kubeflow.org/v1alpha1" +kind: PodDefault +metadata: + name: kaggle-access +spec: + selector: + matchLabels: + kaggle-secret: "true" + desc: "kaggle-access" + volumeMounts: + - name: secret-volume + mountPath: /secret/kaggle + volumes: + - name: secret-volume + secret: + secretName: kaggle-secret +``` +Screenshot 2022-07-04 at 4 56 41 PM + +- Apply the resource.yaml file: `kubectl apply -f resource.yaml` + +## Step 9: Explore the pre-process directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/pre-process` directory +- Open up the `preprocess.py` file +- Note the code in this file that will perform the actions required in the “preprocess-data” pipeline step + +Screenshot 2022-07-04 at 5 00 01 PM + +## Step 10: Build the preprocess-data Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/pre-process` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` +## Step 11: Push the preprocess-data Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/load-data` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` +## Step 12: Explore the featureengineering directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/featureengineering` directory +- Open up the `featureengg.py` file +- Note the code in this file that will perform the actions required in the “featureengineering” pipeline step + +Screenshot 2022-07-04 at 5 02 50 PM + +## Step 13: Build the featureengineering Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/featureengineering` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` +## Step 14: Push the featureengineering Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/featureengineering` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 15: Explore the decisiontree directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/decisiontree` directory +- Open up the `decisiontree.py` file +- Note the code in this file that will perform the actions required in the “decision-tree” pipeline step + +Screenshot 2022-07-04 at 5 05 43 PM + +## Step 16: Build the decisiontree Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/decisiontree` directory +Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 17: Push the decisiontree Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/decisiontree` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` +## Step 18: Explore the logisticregression directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/logisticregression` directory +- Open up the `regression.py` file +- Note the code in this file that will perform the actions required in the “regression” pipeline step + +Screenshot 2022-07-04 at 5 08 11 PM + +## Step 19: Build the regression Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/logisticregression` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` +## Step 20: Push the regression Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/logisticregression` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` +## Step 21: Explore the naivebayes directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/naivebayes` directory +- Open up the `naivebayes.py` file +Note the code in this file that will perform the actions required in the “bayes” pipeline step +Screenshot 2022-07-04 at 5 10 36 PM + +## Step 22: Build the naivebayes Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/naivebayes` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` +## Step 23: Push the naivebayes Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/naivebayes` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` +## Step 24: Explore the randomforest directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/randomforest` directory +- Open up the `randomforest.py` file +- Note the code in this file that will perform the actions required in the “random-forest” pipeline step + + +Screenshot 2022-07-04 at 5 12 54 PM + +## Step 25: Build the random-forest Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/randomforest` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 26: Push the random-forest Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/randomforest` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` +## Step 27: Explore the svm directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/svm` directory +- Open up the `svm.py` file +- Note the code in this file that will perform the actions required in the “svm” pipeline step + +Screenshot 2022-07-04 at 5 15 23 PM + +## Step 28: Build the svm Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/svm` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` +## Step 29: Push the svm Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/svm` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` +## Step 30: Explore the results directory + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/results` directory +- Open up the `result.py` file +- Note the code in this file that will perform the actions required in the “results” pipeline step + +Screenshot 2022-07-04 at 5 18 34 PM + +## Step 31: Build the results Docker Image + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/results` directory +- Build the Docker image if locally you are using arm64 (Apple M1) +``` +docker build --platform=linux/amd64 -t /:-amd64 . +``` +- OR build the Docker image if locally you are using amd64 +``` +docker build -t /: . +``` + +## Step 32: Push the results Docker Image to DockerHub + +- (Locally) Navigate to the `titanic-kaggle-competition/pipeline-components/results` directory +- Push the Docker image if locally you are using arm64 (Apple M1) +``` +docker push /:-amd64 +``` +- OR build the Docker image if locally you are using amd64 +``` +docker push /: +``` + +## Step 33: Modify the titanic-kfp.py file + +- (Kubeflow as a Service) Navigate to the `titanic-kaggle-competition` directory +- Update the `titanic-kfp.py` with accurate Docker Image inputs +``` + return dsl.ContainerOp( + name = 'Preprocess Data', + image = '/:', + +—----- + + return dsl.ContainerOp( + name='featureengineering', + image = '/:', + +—----- + + return dsl.ContainerOp( + name='regression', + image = '/:', + +—----- + + return dsl.ContainerOp( + name='bayes', + image = '/:', + +—----- + + return dsl.ContainerOp( + name='random_forest', + image = '/:', + +—----- + + return dsl.ContainerOp( + name='decision_tree', + image = '/:', + +—----- + + return dsl.ContainerOp( + name='svm', + image = '/:', + +—----- + + return dsl.ContainerOp( + name='results', + image = '/:', +``` + +## Step 34: Generate a KFP Pipeline yaml File + +- (Kubeflow as a Service) Navigate to the `titanic-kaggle-competition` directory +Build a python virtual environment: + +Step a) Update pip +``` +python3 -m pip install --upgrade pip +``` + +Step b) Install virtualenv +``` +sudo pip3 install virtualenv +``` + +Step c) Check the installed version of venv +``` +virtualenv --version +``` + +Step d) Name your virtual enviornment as kfp +``` +virtualenv kfp +``` + +Step e) Activate your venv. +``` +source kfp/bin/activate +``` + +After this virtual environment will get activated. Now in our activated venv we need to install following packages: +``` +sudo apt-get update +sudo apt-get upgrade +sudo apt-get install -y git python3-pip + +python3 -m pip install kfp==1.1.2 +``` + +After installing packages create the yaml file +``` +python3 titanic-kaggle-competition-kfp.py +``` + +Screenshot 2022-07-04 at 5 27 37 PM + + +Download the `titanic-kaggle-competition-kfp.yaml` file that was created to your local `titanic-kaggle-competition` directory. + +## Step 35: Create an Experiment + +- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Experiments (KFP) > Create Experiment view +- Name the experiment and click Next +- Click on Experiments (KFP) to view the experiment you just created + +## Step 36: Create a Pipeline + +- (Kubeflow as a Service) Within the Kubeflow Central Dashboard, navigate to the Pipelines > +Upload Pipeline view +- Name the pipeline +- Click on Upload a file +- Upload the local `titanic-kaggle-competition-kfp.yaml` file +- Click Create + + +## Step 37: Create a Run + +- (Kubeflow as a Service) Click on Create Run in the view from the previous step +- Choose the experiment we created in Step 35 +- Click Start +- Click on the run name to view the runtime execution graph + + +![image10](https://user-images.githubusercontent.com/17012391/177150882-3c8abf80-2d6e-4467-9b11-7824d3909e35.png) + + +## Troubleshooting Tips: +While running the pipeline as mentioned above you may come across this error: +errorlog: + +``` +kaggle.rest.ApiException: (403) +Reason: Forbidden +HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Thu, 23 Jun 2022 11:31:18 GMT', 'Access-Control-Allow-Credentials': 'true', 'Set-Cookie': 'ka_sessionid=6817a347c75399a531148e19cad0aaeb; max-age=2626560; path=/, GCLB=CIGths3--ebbUg; path=/; HttpOnly', 'Transfer-Encoding': 'chunked', 'Vary': +HTTP response body: b'{"code":403,"message":"You must accept this competition\\u0027s rules before you\\u0027ll be able to download files."}' + +``` +This error occours for two reasons: +- Your Kaggle account is not verified with your phone number. +- Rules for this specific competitions are not accepted. + +A solution to this is please verify your Kaggle account using your phone number and accept the rules for this specific competition, untill these two steps are satisfied pipeline wont accquire data from Kaggle API and it wont run.