[{"data":1,"prerenderedAt":4253},["ShallowReactive",2],{"article_list_jbagley_":3},[4,408,656,1651,1796],{"_path":5,"_dir":6,"_draft":7,"_partial":7,"_locale":8,"title":9,"description":10,"publishDate":11,"image":12,"tags":13,"excerpt":10,"body":18,"_type":399,"_id":400,"_source":401,"_file":402,"_stem":403,"_extension":404,"author":405},"/jbagley/2025-08/a_developers_primer_on_apple_tracking_transparency","2025-08",false,"","A Primer on Apple's App Tracking Transparency","If an app tracks user activity, Apple requires them to declare all information they collect as well as whether that data is linked or tracked. This includes collection by the app itself and any third parties the app uses. The app owner is responsible for knowing and correctly reporting privacy information for all components in the app.","2026-05-22","/jbagley/2025-08/img/apple_app_transparency.png",[14,15,16,17],"app tracking transparency","att","ios","macos",{"type":19,"children":20,"toc":386},"root",[21,30,48,53,58,65,75,85,95,104,120,126,138,143,149,171,176,181,187,193,198,203,209,217,234,249,262,268,273,284,297,302,317,323,328,342,348,361,366,372],{"type":22,"tag":23,"props":24,"children":26},"element","h1",{"id":25},"overview",[27],{"type":28,"value":29},"text","Overview",{"type":22,"tag":31,"props":32,"children":33},"p",{},[34,36,46],{"type":28,"value":35},"If an app tracks user activity, Apple requires them to declare ",{"type":22,"tag":37,"props":38,"children":39},"em",{},[40],{"type":22,"tag":41,"props":42,"children":43},"strong",{},[44],{"type":28,"value":45},"all",{"type":28,"value":47}," information they collect as well as whether that data is linked or tracked. This includes collection by the app itself and any third parties the app uses. The app owner is responsible for knowing and correctly reporting privacy information for all components in the app.",{"type":22,"tag":31,"props":49,"children":50},{},[51],{"type":28,"value":52},"The information is declared in two places, the Privacy Nutrition Label shown with the app in the App Store, and a privacy manifest the app carries. If an app will be tracking users, it must request permission to do so from the user.",{"type":22,"tag":31,"props":54,"children":55},{},[56],{"type":28,"value":57},"Collectively these requirements and practices are known as App Tracking Transparency (ATT).",{"type":22,"tag":59,"props":60,"children":62},"h2",{"id":61},"terms",[63],{"type":28,"value":64},"Terms",{"type":22,"tag":31,"props":66,"children":67},{},[68,73],{"type":22,"tag":41,"props":69,"children":70},{},[71],{"type":28,"value":72},"Collecting",{"type":28,"value":74}," information is storing information for longer than it is needed to perform whatever function generated or required the information. For example, even server logs might be considered collecting.",{"type":22,"tag":31,"props":76,"children":77},{},[78,83],{"type":22,"tag":41,"props":79,"children":80},{},[81],{"type":28,"value":82},"Linking",{"type":28,"value":84}," is connecting the collected data to a specific user in some way.",{"type":22,"tag":31,"props":86,"children":87},{},[88,93],{"type":22,"tag":41,"props":89,"children":90},{},[91],{"type":28,"value":92},"Tracking",{"type":28,"value":94}," is",{"type":22,"tag":96,"props":97,"children":98},"blockquote",{},[99],{"type":22,"tag":31,"props":100,"children":101},{},[102],{"type":28,"value":103},"\"aggregating linked data from one or more sources to build up some profile of a specific user that potentially can be combined with information form other sources to target a user for advertising or tracking advertising performance.\"",{"type":22,"tag":31,"props":105,"children":106},{},[107,109,118],{"type":28,"value":108},"(From ",{"type":22,"tag":110,"props":111,"children":115},"a",{"href":112,"rel":113},"https://developer.apple.com/app-store/user-privacy-and-data-use/",[114],"nofollow",[116],{"type":28,"value":117},"Apple's privacy and data use documentation",{"type":28,"value":119},".)",{"type":22,"tag":59,"props":121,"children":123},{"id":122},"concerns",[124],{"type":28,"value":125},"Concerns",{"type":22,"tag":31,"props":127,"children":128},{},[129,131,136],{"type":28,"value":130},"Determining what to include in these privacy declarations takes all parties. The organization that owns the app ",{"type":22,"tag":37,"props":132,"children":133},{},[134],{"type":28,"value":135},"and their legal counsel",{"type":28,"value":137}," should review the information. What the app declares should match the organization's posted privacy policy, of course.",{"type":22,"tag":31,"props":139,"children":140},{},[141],{"type":28,"value":142},"Code reviews should be done to determine how information is being used and the privacy implications. This might involve reviewing logging and database schemas as well as reviewing where data is transmitted. If a third party performs any data collecting, their privacy statements and manifests should be reviewed.",{"type":22,"tag":23,"props":144,"children":146},{"id":145},"privacy-nutrition-label",[147],{"type":28,"value":148},"Privacy Nutrition Label",{"type":22,"tag":31,"props":150,"children":151},{},[152,154,161,163,169],{"type":28,"value":153},"In 2020 Apple introduced the Privacy Nutrition Label. They are created using App Store Connect. In the ",{"type":22,"tag":155,"props":156,"children":158},"code",{"className":157},[],[159],{"type":28,"value":160},"App Store > TRUST & SAFETY > App Privacy",{"type":28,"value":162}," section under the ",{"type":22,"tag":155,"props":164,"children":166},{"className":165},[],[167],{"type":28,"value":168},"Distribution",{"type":28,"value":170}," tab, Apple provides a UI for creating the label.",{"type":22,"tag":31,"props":172,"children":173},{},[174],{"type":28,"value":175},"I don't recommend starting with this. Instead, create the privacy manifest because it is seen as the ground truth by Apple. After the privacy manifest exists, Xcode can provide a privacy report for your app that makes this easier.",{"type":22,"tag":31,"props":177,"children":178},{},[179],{"type":28,"value":180},"One quirk to mention is that changes to the Privacy Nutrition Label go live on the App Store immediately, so there could be discrepancies when submitting a new version.",{"type":22,"tag":182,"props":183,"children":184},"aside",{},[185],{"type":28,"value":186},"\n*I don't understand why Apple does not automatically populate this from the submitted app's privacy manifest files.*\n",{"type":22,"tag":23,"props":188,"children":190},{"id":189},"tracking-permission",[191],{"type":28,"value":192},"Tracking Permission",{"type":22,"tag":31,"props":194,"children":195},{},[196],{"type":28,"value":197},"In 2021, Apple added the requirement for requesting permission from the user to allow tracking. If the user does not agree, then it is up to the app to make sure no tracking occurs by it or any third party libraries it uses.",{"type":22,"tag":31,"props":199,"children":200},{},[201],{"type":28,"value":202},"Keep in mind that users disable allowing tracking in their OS settings; meaning they will never see a request for permission, and any queries for the permission will return a value meaning not permitted.",{"type":22,"tag":23,"props":204,"children":206},{"id":205},"privacy-manifests",[207],{"type":28,"value":208},"Privacy Manifests",{"type":22,"tag":96,"props":210,"children":211},{},[212],{"type":22,"tag":31,"props":213,"children":214},{},[215],{"type":28,"value":216},"\"A privacy manifest is a property list file (PrivacyInfo.xcprivacy) that you add to your target’s resources. The privacy manifest describes the privacy practices of an app or third-party SDK.\"",{"type":22,"tag":218,"props":219,"children":220},"ul",{},[221],{"type":22,"tag":222,"props":223,"children":224},"li",{},[225,227],{"type":28,"value":226},"From ",{"type":22,"tag":110,"props":228,"children":231},{"href":229,"rel":230},"https://developer.apple.com/documentation/bundleresources/adding-a-privacy-manifest-to-your-app-or-third-party-sdk",[114],[232],{"type":28,"value":233},"Apple's documentation for adding a privacy manifest",{"type":22,"tag":31,"props":235,"children":236},{},[237,239,247],{"type":28,"value":238},"In 2023 Apple introduced privacy manifests, and in 2024 it ",{"type":22,"tag":110,"props":240,"children":244},{"href":241,"rel":242,"title":243},"https://developer.apple.com/news/?id=pvszzano",[114],"Apple's reminder for privacy changes",[245],{"type":28,"value":246},"started to require them",{"type":28,"value":248}," in apps submitted to the App Store.",{"type":22,"tag":31,"props":250,"children":251},{},[252,254,260],{"type":28,"value":253},"Xcode also provides an editor that has the relevant keys and values like it does for the ",{"type":22,"tag":155,"props":255,"children":257},{"className":256},[],[258],{"type":28,"value":259},"info.plist",{"type":28,"value":261}," format.",{"type":22,"tag":59,"props":263,"children":265},{"id":264},"generating-the-manifest",[266],{"type":28,"value":267},"Generating the manifest",{"type":22,"tag":31,"props":269,"children":270},{},[271],{"type":28,"value":272},"Create a privacy manifest using",{"type":22,"tag":274,"props":275,"children":279},"pre",{"className":276,"code":278,"language":28},[277],"language-text","New > File from template...\n",[280],{"type":22,"tag":155,"props":281,"children":282},{"__ignoreMap":8},[283],{"type":28,"value":278},{"type":22,"tag":31,"props":285,"children":286},{},[287,289,295],{"type":28,"value":288},"and choose the template ",{"type":22,"tag":155,"props":290,"children":292},{"className":291},[],[293],{"type":28,"value":294},"App Privacy",{"type":28,"value":296},".",{"type":22,"tag":31,"props":298,"children":299},{},[300],{"type":28,"value":301},"It's then a painstaking, manual process to input what data is collected and how it is used. Referring to the app's privacy policy statement if it already exists is the best way to populate this file. Ideally, the organizational review and code reviews will have been done by the time you create this.",{"type":22,"tag":182,"props":303,"children":304},{},[305,312],{"type":22,"tag":306,"props":307,"children":309},"h3",{"id":308},"third-party-software",[310],{"type":28,"value":311},"Third Party Software",{"type":22,"tag":31,"props":313,"children":314},{},[315],{"type":28,"value":316},"Apple encourages library providers to include privacy manifest files. Some that Apple considers critical to privacy–like Firebase, Meta or Branch–are required to as well as being signed. I found every library my app uses to be compliant, but you should be aware of it.",{"type":22,"tag":59,"props":318,"children":320},{"id":319},"tracking-domains",[321],{"type":28,"value":322},"Tracking Domains",{"type":22,"tag":31,"props":324,"children":325},{},[326],{"type":28,"value":327},"The manifest's tracking domain section lets you black list some domains used for tracking. This is a fail-safe to prevent tracking if the user has not agreed to it. These domains will be blocked automatically when the user does not want to be tracked.",{"type":22,"tag":182,"props":329,"children":330},{},[331,337],{"type":22,"tag":306,"props":332,"children":334},{"id":333},"verifying",[335],{"type":28,"value":336},"Verifying",{"type":22,"tag":31,"props":338,"children":339},{},[340],{"type":28,"value":341},"Xcode's Instruments can build a profile of network connections during an app run, allowing you to verify which domains your app contacts.",{"type":22,"tag":59,"props":343,"children":345},{"id":344},"checking-your-work",[346],{"type":28,"value":347},"Checking your work",{"type":22,"tag":31,"props":349,"children":350},{},[351,353,359],{"type":28,"value":352},"To see your app's privacy manifest in a user friendly way, use Xcode Organizer. Generate an archive build, then right click on the arvie in the Organizer and select ",{"type":22,"tag":155,"props":354,"children":356},{"className":355},[],[357],{"type":28,"value":358},"Generate Privacy Report",{"type":28,"value":360},". This creates a PDF with the information about data types collected and how they are used. It will include the third party privacy manifests information.",{"type":22,"tag":31,"props":362,"children":363},{},[364],{"type":28,"value":365},"Now use this report to create the Privacy Nutrition Label in App Store Connect.",{"type":22,"tag":23,"props":367,"children":369},{"id":368},"conclusion",[370],{"type":28,"value":371},"Conclusion",{"type":22,"tag":31,"props":373,"children":374},{},[375,377,384],{"type":28,"value":376},"Hopefully this primer will keep your head from spinning too much when you get more into ",{"type":22,"tag":110,"props":378,"children":381},{"href":379,"rel":380},"https://developer.apple.com/app-store/app-privacy-details/",[114],[382],{"type":28,"value":383},"Apple's comprehensive overview",{"type":28,"value":385},". Good luck!",{"title":8,"searchDepth":387,"depth":387,"links":388},3,[389,391,392,395,398],{"id":61,"depth":390,"text":64},2,{"id":122,"depth":390,"text":125},{"id":264,"depth":390,"text":267,"children":393},[394],{"id":308,"depth":387,"text":311},{"id":319,"depth":390,"text":322,"children":396},[397],{"id":333,"depth":387,"text":336},{"id":344,"depth":390,"text":347},"markdown","content:jbagley:2025-08:a_developers_primer_on_apple_tracking_transparency.md","content","jbagley/2025-08/a_developers_primer_on_apple_tracking_transparency.md","jbagley/2025-08/a_developers_primer_on_apple_tracking_transparency","md",{"user":406,"name":407},"jbagley","Jason Bagley",{"_path":409,"_dir":410,"_draft":7,"_partial":7,"_locale":8,"title":411,"description":412,"tags":413,"excerpt":412,"image":418,"publishDate":419,"body":420,"_type":399,"_id":652,"_source":401,"_file":653,"_stem":654,"_extension":404,"author":655},"/jbagley/2023-06-01/universal_ffmpeg_custom_builds","2023-06-01","Building Universal FFmpeg Custom Binaries","I am using a very pared down set of FFMpeg features for a macOS project that I\nbuild into a custom library. I had a script set up to configure the build which\nworked fine on my Intel based MacBook Pro. Then I upgraded to an Apple Silicon\nMacBookPro and wanted to run natively, or at least see what happened when I\ndid. To build, FFMpeg uses autoconf which produces a makefile that then handles\nthe build.",[414,415,416,417],"c","bash","ffmpeg","apple","/jbagley/2023-06-01/img/header.png","2024-04-01",{"type":19,"children":421,"toc":646},[422,428,432,438,468,481,487,492,497,506,511,548,569,575,595,604,616,628,641],{"type":22,"tag":59,"props":423,"children":425},{"id":424},"motivation",[426],{"type":28,"value":427},"Motivation",{"type":22,"tag":31,"props":429,"children":430},{},[431],{"type":28,"value":412},{"type":22,"tag":59,"props":433,"children":435},{"id":434},"the-simplest-thing-which-could-work-but-didnt",[436],{"type":28,"value":437},"The simplest thing which could work but didn't",{"type":22,"tag":31,"props":439,"children":440},{},[441,443,449,451,457,459,466],{"type":28,"value":442},"Autoconf supports passing flags for the compiler in the call to configure. I was\nalready passing ",{"type":22,"tag":155,"props":444,"children":446},{"className":445},[],[447],{"type":28,"value":448},"-arch x86_64",{"type":28,"value":450}," as part of ",{"type":22,"tag":155,"props":452,"children":454},{"className":453},[],[455],{"type":28,"value":456},"--extra-cflags",{"type":28,"value":458},". Its\n",{"type":22,"tag":110,"props":460,"children":463},{"href":461,"rel":462},"https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Multiple-Architectures.html",[114],[464],{"type":28,"value":465},"documentation",{"type":28,"value":467},"\nsays that I may provide more than one architecture on macOS. That page notes\nthat this doesn't always work.",{"type":22,"tag":31,"props":469,"children":470},{},[471,473,479],{"type":28,"value":472},"Based on that, I tried ",{"type":22,"tag":155,"props":474,"children":476},{"className":475},[],[477],{"type":28,"value":478},"-arch x86_64 -arch arm64",{"type":28,"value":480},"', and as foreshadowed it\nindeed did not work. Autoconf's compiler tests would fail when trying the x86_64\nbuild with clang. I went through some more permutations of arguments, but in the\nend I had to do separate builds for each architecture.",{"type":22,"tag":59,"props":482,"children":484},{"id":483},"building",[485],{"type":28,"value":486},"Building",{"type":22,"tag":31,"props":488,"children":489},{},[490],{"type":28,"value":491},"I use a build directory rather than calling configure in the top of my FFmpeg\nworking copy and dirtying it up. I am running this from that build dir.",{"type":22,"tag":31,"props":493,"children":494},{},[495],{"type":28,"value":496},"Here is bash psuedocode of my build script:",{"type":22,"tag":274,"props":498,"children":501},{"className":499,"code":500,"language":28},[277],"for arch in x86_64 arm64; do\n\n   rm -rf Makefile config.h ffbuild lib* tests \n\n    ../configure --cc=clang \\\n                 --extra-cflags=\"-arch $arch\" \\\n                 --extra-ldflags=\"-arch $arch\"  \\\n                 --build_suffix=\"$install_dir\"  \\\n                 --prefix=\"path/to/where/I/want/the/libs/to/be/copied/by/make/install\" \\\n                 # other flags for ffmpeg configuration, see '../configure -h' output\n    make -j\n    make install\ndone\n",[502],{"type":22,"tag":155,"props":503,"children":504},{"__ignoreMap":8},[505],{"type":28,"value":500},{"type":22,"tag":31,"props":507,"children":508},{},[509],{"type":28,"value":510},"Looping through the desired architectures, first it cleans the build directory\nto get rid of the previous architecture output, then configures, building and\ninstalling the library.",{"type":22,"tag":31,"props":512,"children":513},{},[514,516,522,524,530,532,538,540,546],{"type":28,"value":515},"In the end, you have two sets of library files installed into the path passed\nwith ",{"type":22,"tag":155,"props":517,"children":519},{"className":518},[],[520],{"type":28,"value":521},"--prefix",{"type":28,"value":523}," with the suffixes ",{"type":22,"tag":155,"props":525,"children":527},{"className":526},[],[528],{"type":28,"value":529},"_x86-64",{"type":28,"value":531}," and ",{"type":22,"tag":155,"props":533,"children":535},{"className":534},[],[536],{"type":28,"value":537},"_arm64",{"type":28,"value":539}," provided by the\n",{"type":22,"tag":155,"props":541,"children":543},{"className":542},[],[544],{"type":28,"value":545},"--build_suffix",{"type":28,"value":547}," argument.",{"type":22,"tag":31,"props":549,"children":550},{},[551,553,559,561,567],{"type":28,"value":552},"I needed to pass the architecture again with ",{"type":22,"tag":155,"props":554,"children":556},{"className":555},[],[557],{"type":28,"value":558},"--extra-ldflags",{"type":28,"value":560}," to avoid the\nautoconf compiler test failing. The most basic test for compiler support would\nfail when an x86_64 ",{"type":22,"tag":155,"props":562,"children":564},{"className":563},[],[565],{"type":28,"value":566},".o",{"type":28,"value":568}," file couldn't be linked to an arm64 exectuable.",{"type":22,"tag":59,"props":570,"children":572},{"id":571},"linking-the-universal-libraries",[573],{"type":28,"value":574},"Linking The Universal Libraries",{"type":22,"tag":31,"props":576,"children":577},{},[578,580,586,588,593],{"type":28,"value":579},"The call to ",{"type":22,"tag":155,"props":581,"children":583},{"className":582},[],[584],{"type":28,"value":585},"lipo",{"type":28,"value":587}," turned out to be simple. The trickier part for me was\nbuilding the array of names to pass to ",{"type":22,"tag":155,"props":589,"children":591},{"className":590},[],[592],{"type":28,"value":585},{"type":28,"value":594}," as input files. I don't remember\nthe last time I used these.",{"type":22,"tag":274,"props":596,"children":599},{"className":597,"code":598,"language":28},[277],"lib_output_dir=\u003Cthe same path I used as the --prefix argument to configure above>\ncd \"$lib_output_dir\"\nfor lib_name in \u003Clib names I produced in my build without the file extension, e.g. libavcodec>; do\n    input_names=()\n    for arch in x86_64 arm64; do\n        input_names+=(${lib_name}_{$arch}.a)\n    done\n    /usr/bin/lipo -create -output ${lib_name}.a ${input_names[*]}\n    rm ${input_names}[*]}\ndone\ncd -\n",[600],{"type":22,"tag":155,"props":601,"children":602},{"__ignoreMap":8},[603],{"type":28,"value":598},{"type":22,"tag":31,"props":605,"children":606},{},[607,609,614],{"type":28,"value":608},"It changes to the directory where the first section installed the libraries. Of\ncourse, ",{"type":22,"tag":155,"props":610,"children":612},{"className":611},[],[613],{"type":28,"value":585},{"type":28,"value":615}," could be run on the libs within the build folder, but my script\nwas already installing them, and I found this more convenient and simple.",{"type":22,"tag":31,"props":617,"children":618},{},[619,621,626],{"type":28,"value":620},"For each architecture, it builds the array of library filenames, including the\narchitecture suffix of each. That array gives the typical DRY advantages and\nkeeps the invocation of ",{"type":22,"tag":155,"props":622,"children":624},{"className":623},[],[625],{"type":28,"value":585},{"type":28,"value":627}," simple.",{"type":22,"tag":31,"props":629,"children":630},{},[631,633,639],{"type":28,"value":632},"You can confirm it worked using the ",{"type":22,"tag":155,"props":634,"children":636},{"className":635},[],[637],{"type":28,"value":638},"file",{"type":28,"value":640}," utility. You should see a line for each\narchitecture it its output.",{"type":22,"tag":31,"props":642,"children":643},{},[644],{"type":28,"value":645},"That's it. If you have a better way, or see something I can do better here, let\nme know.",{"title":8,"searchDepth":387,"depth":387,"links":647},[648,649,650,651],{"id":424,"depth":390,"text":427},{"id":434,"depth":390,"text":437},{"id":483,"depth":390,"text":486},{"id":571,"depth":390,"text":574},"content:jbagley:2023-06-01:Universal_FFMPEG_custom_builds.md","jbagley/2023-06-01/Universal_FFMPEG_custom_builds.md","jbagley/2023-06-01/Universal_FFMPEG_custom_builds",{"user":406,"name":407},{"_path":657,"_dir":658,"_draft":7,"_partial":7,"_locale":8,"title":659,"description":660,"tags":661,"image":664,"publishDate":658,"excerpt":660,"body":665,"_type":399,"_id":1647,"_source":401,"_file":1648,"_stem":1649,"_extension":404,"author":1650},"/jbagley/2021-08-01/accuratetiming","2021-08-01","Accurate Timing","In many tasks we need to do something at given intervals of time. The most obvious ways may not give you the best results.",[662,663],"c++","timing","/jbagley/2021-08-01/img/accurateTiming.jpg",{"type":19,"children":666,"toc":1640},[667,671,677,690,826,847,852,858,863,881,1518,1537,1550,1555,1564,1577,1611,1622,1634],{"type":22,"tag":31,"props":668,"children":669},{},[670],{"type":28,"value":660},{"type":22,"tag":59,"props":672,"children":674},{"id":673},"time-meh",[675],{"type":28,"value":676},"Time? Meh.",{"type":22,"tag":31,"props":678,"children":679},{},[680,682,688],{"type":28,"value":681},"The most basic tasks that don't have what you might call CPU-scale time\nrequirements can be handled with the usual language and framework features. These\ncan be things like timers, or simply a call to a ",{"type":22,"tag":155,"props":683,"children":685},{"className":684},[],[686],{"type":28,"value":687},"sleep(number_of_milliseconds)",{"type":28,"value":689}," function.",{"type":22,"tag":274,"props":691,"children":695},{"className":692,"code":693,"language":694,"meta":8,"style":8},"language-cpp shiki shiki-themes github-light github-dark","void RunTimedThing()\n{\n   while (true)\n   {\n      DoMyThing();\n      sleep(1);      // wake up about 1 millisecond later to do it again.\n   }\n}\n","cpp",[696],{"type":22,"tag":155,"props":697,"children":698},{"__ignoreMap":8},[699,723,731,755,764,778,808,817],{"type":22,"tag":700,"props":701,"children":704},"span",{"class":702,"line":703},"line",1,[705,711,717],{"type":22,"tag":700,"props":706,"children":708},{"style":707},"--shiki-default:#D73A49;--shiki-dark:#F97583",[709],{"type":28,"value":710},"void",{"type":22,"tag":700,"props":712,"children":714},{"style":713},"--shiki-default:#6F42C1;--shiki-dark:#B392F0",[715],{"type":28,"value":716}," RunTimedThing",{"type":22,"tag":700,"props":718,"children":720},{"style":719},"--shiki-default:#24292E;--shiki-dark:#E1E4E8",[721],{"type":28,"value":722},"()\n",{"type":22,"tag":700,"props":724,"children":725},{"class":702,"line":390},[726],{"type":22,"tag":700,"props":727,"children":728},{"style":719},[729],{"type":28,"value":730},"{\n",{"type":22,"tag":700,"props":732,"children":733},{"class":702,"line":387},[734,739,744,750],{"type":22,"tag":700,"props":735,"children":736},{"style":707},[737],{"type":28,"value":738},"   while",{"type":22,"tag":700,"props":740,"children":741},{"style":719},[742],{"type":28,"value":743}," (",{"type":22,"tag":700,"props":745,"children":747},{"style":746},"--shiki-default:#005CC5;--shiki-dark:#79B8FF",[748],{"type":28,"value":749},"true",{"type":22,"tag":700,"props":751,"children":752},{"style":719},[753],{"type":28,"value":754},")\n",{"type":22,"tag":700,"props":756,"children":758},{"class":702,"line":757},4,[759],{"type":22,"tag":700,"props":760,"children":761},{"style":719},[762],{"type":28,"value":763},"   {\n",{"type":22,"tag":700,"props":765,"children":767},{"class":702,"line":766},5,[768,773],{"type":22,"tag":700,"props":769,"children":770},{"style":713},[771],{"type":28,"value":772},"      DoMyThing",{"type":22,"tag":700,"props":774,"children":775},{"style":719},[776],{"type":28,"value":777},"();\n",{"type":22,"tag":700,"props":779,"children":781},{"class":702,"line":780},6,[782,787,792,797,802],{"type":22,"tag":700,"props":783,"children":784},{"style":713},[785],{"type":28,"value":786},"      sleep",{"type":22,"tag":700,"props":788,"children":789},{"style":719},[790],{"type":28,"value":791},"(",{"type":22,"tag":700,"props":793,"children":794},{"style":746},[795],{"type":28,"value":796},"1",{"type":22,"tag":700,"props":798,"children":799},{"style":719},[800],{"type":28,"value":801},");",{"type":22,"tag":700,"props":803,"children":805},{"style":804},"--shiki-default:#6A737D;--shiki-dark:#6A737D",[806],{"type":28,"value":807},"      // wake up about 1 millisecond later to do it again.\n",{"type":22,"tag":700,"props":809,"children":811},{"class":702,"line":810},7,[812],{"type":22,"tag":700,"props":813,"children":814},{"style":719},[815],{"type":28,"value":816},"   }\n",{"type":22,"tag":700,"props":818,"children":820},{"class":702,"line":819},8,[821],{"type":22,"tag":700,"props":822,"children":823},{"style":719},[824],{"type":28,"value":825},"}\n",{"type":22,"tag":96,"props":827,"children":828},{},[829],{"type":22,"tag":31,"props":830,"children":831},{},[832,837,839,845],{"type":22,"tag":41,"props":833,"children":834},{},[835],{"type":28,"value":836},"Exercise 1",{"type":28,"value":838},": How much time is there between each call to ",{"type":22,"tag":155,"props":840,"children":842},{"className":841},[],[843],{"type":28,"value":844},"DoMyThing()",{"type":28,"value":846},"?",{"type":22,"tag":31,"props":848,"children":849},{},[850],{"type":28,"value":851},"Most OS thread scheduling is not guaranteed, so your thread isn't usually\nexecuted again exactly when you want it to be. If you are fine with your thing\nbeing done again not exactly 1 ms later, this approach is acceptable. But if you\nneed accuracy to the millisecond, or you need it to run more frequently than 1\nms, it is time to level up.",{"type":22,"tag":59,"props":853,"children":855},{"id":854},"when-it-positively-absolutely-has-to-be-done-on-time",[856],{"type":28,"value":857},"When it positively, absolutely has to be done on time",{"type":22,"tag":31,"props":859,"children":860},{},[861],{"type":28,"value":862},"Now you will need a high resolution timer. These have millisecond or even\nnanosecond accuracy. It used to be more of a _maybe-you-can-maybe-you-can't kind\nof feature feature, but with modern hardware, even the small guys, you usually\nwill be able to get all the way up to nanosecond precision.",{"type":22,"tag":31,"props":864,"children":865},{},[866,868,879],{"type":28,"value":867},"For C++, ",{"type":22,"tag":110,"props":869,"children":872},{"href":870,"rel":871},"https://en.cppreference.com/w/cpp/chrono",[114],[873],{"type":22,"tag":155,"props":874,"children":876},{"className":875},[],[877],{"type":28,"value":878},"std::chrono",{"type":28,"value":880}," provides such facilities.",{"type":22,"tag":274,"props":882,"children":884},{"className":692,"code":883,"language":694,"meta":8,"style":8},"using namespace std::chrono_literals;              // For using 1ms as a const literal for 1 millisecond\nusing namespace std::chrono;                       // For the sake of brevity and sanity\nusing Clock = std::chrono::high_resolution_clock;  // For ease in changing our timing resolution\n\nvoid RunTimedThingAccurately()\n{\n   const auto kMyThingFrequency = 1ms;\n   const auto kMaxSleepTime = kMyThingFrequency / 2.0; // set to something less than the frequency\n\n   auto nextTime = Clock::now();\n   while (true)\n   {\n      const auto currentTime = Clock::now();\n      const auto remainingTime = duration_cast\u003Cmicroseconds>(nextTime - currentTime);\n      if (remainingTime.count() \u003C= 0LL)\n      {\n         DoMyThing();\n         nextTime = currentTime + kMyThingFrequency + remainingTime;\n         sleepTime = maxSleepTime;\n      }\n      else\n      {\n         sleepTime = std::min(maxSleepTime, remainingTime);\n      }\n  \n      std::this_thread::sleep_for(remainingTime);\n   }\n}\n",[885],{"type":22,"tag":155,"props":886,"children":887},{"__ignoreMap":8},[888,926,959,1006,1015,1031,1038,1076,1119,1127,1162,1182,1190,1228,1279,1322,1331,1344,1379,1397,1406,1415,1423,1453,1461,1470,1502,1510],{"type":22,"tag":700,"props":889,"children":890},{"class":702,"line":703},[891,896,901,906,911,916,921],{"type":22,"tag":700,"props":892,"children":893},{"style":707},[894],{"type":28,"value":895},"using",{"type":22,"tag":700,"props":897,"children":898},{"style":707},[899],{"type":28,"value":900}," namespace",{"type":22,"tag":700,"props":902,"children":903},{"style":713},[904],{"type":28,"value":905}," std",{"type":22,"tag":700,"props":907,"children":908},{"style":719},[909],{"type":28,"value":910},"::",{"type":22,"tag":700,"props":912,"children":913},{"style":713},[914],{"type":28,"value":915},"chrono_literals",{"type":22,"tag":700,"props":917,"children":918},{"style":719},[919],{"type":28,"value":920},";",{"type":22,"tag":700,"props":922,"children":923},{"style":804},[924],{"type":28,"value":925},"              // For using 1ms as a const literal for 1 millisecond\n",{"type":22,"tag":700,"props":927,"children":928},{"class":702,"line":390},[929,933,937,941,945,950,954],{"type":22,"tag":700,"props":930,"children":931},{"style":707},[932],{"type":28,"value":895},{"type":22,"tag":700,"props":934,"children":935},{"style":707},[936],{"type":28,"value":900},{"type":22,"tag":700,"props":938,"children":939},{"style":713},[940],{"type":28,"value":905},{"type":22,"tag":700,"props":942,"children":943},{"style":719},[944],{"type":28,"value":910},{"type":22,"tag":700,"props":946,"children":947},{"style":713},[948],{"type":28,"value":949},"chrono",{"type":22,"tag":700,"props":951,"children":952},{"style":719},[953],{"type":28,"value":920},{"type":22,"tag":700,"props":955,"children":956},{"style":804},[957],{"type":28,"value":958},"                       // For the sake of brevity and sanity\n",{"type":22,"tag":700,"props":960,"children":961},{"class":702,"line":387},[962,966,971,976,980,984,988,992,997,1001],{"type":22,"tag":700,"props":963,"children":964},{"style":707},[965],{"type":28,"value":895},{"type":22,"tag":700,"props":967,"children":968},{"style":713},[969],{"type":28,"value":970}," Clock",{"type":22,"tag":700,"props":972,"children":973},{"style":707},[974],{"type":28,"value":975}," =",{"type":22,"tag":700,"props":977,"children":978},{"style":713},[979],{"type":28,"value":905},{"type":22,"tag":700,"props":981,"children":982},{"style":719},[983],{"type":28,"value":910},{"type":22,"tag":700,"props":985,"children":986},{"style":713},[987],{"type":28,"value":949},{"type":22,"tag":700,"props":989,"children":990},{"style":719},[991],{"type":28,"value":910},{"type":22,"tag":700,"props":993,"children":994},{"style":713},[995],{"type":28,"value":996},"high_resolution_clock",{"type":22,"tag":700,"props":998,"children":999},{"style":719},[1000],{"type":28,"value":920},{"type":22,"tag":700,"props":1002,"children":1003},{"style":804},[1004],{"type":28,"value":1005},"  // For ease in changing our timing resolution\n",{"type":22,"tag":700,"props":1007,"children":1008},{"class":702,"line":757},[1009],{"type":22,"tag":700,"props":1010,"children":1012},{"emptyLinePlaceholder":1011},true,[1013],{"type":28,"value":1014},"\n",{"type":22,"tag":700,"props":1016,"children":1017},{"class":702,"line":766},[1018,1022,1027],{"type":22,"tag":700,"props":1019,"children":1020},{"style":707},[1021],{"type":28,"value":710},{"type":22,"tag":700,"props":1023,"children":1024},{"style":713},[1025],{"type":28,"value":1026}," RunTimedThingAccurately",{"type":22,"tag":700,"props":1028,"children":1029},{"style":719},[1030],{"type":28,"value":722},{"type":22,"tag":700,"props":1032,"children":1033},{"class":702,"line":780},[1034],{"type":22,"tag":700,"props":1035,"children":1036},{"style":719},[1037],{"type":28,"value":730},{"type":22,"tag":700,"props":1039,"children":1040},{"class":702,"line":810},[1041,1046,1051,1056,1061,1066,1071],{"type":22,"tag":700,"props":1042,"children":1043},{"style":707},[1044],{"type":28,"value":1045},"   const",{"type":22,"tag":700,"props":1047,"children":1048},{"style":707},[1049],{"type":28,"value":1050}," auto",{"type":22,"tag":700,"props":1052,"children":1053},{"style":719},[1054],{"type":28,"value":1055}," kMyThingFrequency ",{"type":22,"tag":700,"props":1057,"children":1058},{"style":707},[1059],{"type":28,"value":1060},"=",{"type":22,"tag":700,"props":1062,"children":1063},{"style":746},[1064],{"type":28,"value":1065}," 1",{"type":22,"tag":700,"props":1067,"children":1068},{"style":707},[1069],{"type":28,"value":1070},"ms",{"type":22,"tag":700,"props":1072,"children":1073},{"style":719},[1074],{"type":28,"value":1075},";\n",{"type":22,"tag":700,"props":1077,"children":1078},{"class":702,"line":819},[1079,1083,1087,1092,1096,1100,1105,1110,1114],{"type":22,"tag":700,"props":1080,"children":1081},{"style":707},[1082],{"type":28,"value":1045},{"type":22,"tag":700,"props":1084,"children":1085},{"style":707},[1086],{"type":28,"value":1050},{"type":22,"tag":700,"props":1088,"children":1089},{"style":719},[1090],{"type":28,"value":1091}," kMaxSleepTime ",{"type":22,"tag":700,"props":1093,"children":1094},{"style":707},[1095],{"type":28,"value":1060},{"type":22,"tag":700,"props":1097,"children":1098},{"style":719},[1099],{"type":28,"value":1055},{"type":22,"tag":700,"props":1101,"children":1102},{"style":707},[1103],{"type":28,"value":1104},"/",{"type":22,"tag":700,"props":1106,"children":1107},{"style":746},[1108],{"type":28,"value":1109}," 2.0",{"type":22,"tag":700,"props":1111,"children":1112},{"style":719},[1113],{"type":28,"value":920},{"type":22,"tag":700,"props":1115,"children":1116},{"style":804},[1117],{"type":28,"value":1118}," // set to something less than the frequency\n",{"type":22,"tag":700,"props":1120,"children":1122},{"class":702,"line":1121},9,[1123],{"type":22,"tag":700,"props":1124,"children":1125},{"emptyLinePlaceholder":1011},[1126],{"type":28,"value":1014},{"type":22,"tag":700,"props":1128,"children":1130},{"class":702,"line":1129},10,[1131,1136,1141,1145,1149,1153,1158],{"type":22,"tag":700,"props":1132,"children":1133},{"style":707},[1134],{"type":28,"value":1135},"   auto",{"type":22,"tag":700,"props":1137,"children":1138},{"style":719},[1139],{"type":28,"value":1140}," nextTime ",{"type":22,"tag":700,"props":1142,"children":1143},{"style":707},[1144],{"type":28,"value":1060},{"type":22,"tag":700,"props":1146,"children":1147},{"style":713},[1148],{"type":28,"value":970},{"type":22,"tag":700,"props":1150,"children":1151},{"style":719},[1152],{"type":28,"value":910},{"type":22,"tag":700,"props":1154,"children":1155},{"style":713},[1156],{"type":28,"value":1157},"now",{"type":22,"tag":700,"props":1159,"children":1160},{"style":719},[1161],{"type":28,"value":777},{"type":22,"tag":700,"props":1163,"children":1165},{"class":702,"line":1164},11,[1166,1170,1174,1178],{"type":22,"tag":700,"props":1167,"children":1168},{"style":707},[1169],{"type":28,"value":738},{"type":22,"tag":700,"props":1171,"children":1172},{"style":719},[1173],{"type":28,"value":743},{"type":22,"tag":700,"props":1175,"children":1176},{"style":746},[1177],{"type":28,"value":749},{"type":22,"tag":700,"props":1179,"children":1180},{"style":719},[1181],{"type":28,"value":754},{"type":22,"tag":700,"props":1183,"children":1185},{"class":702,"line":1184},12,[1186],{"type":22,"tag":700,"props":1187,"children":1188},{"style":719},[1189],{"type":28,"value":763},{"type":22,"tag":700,"props":1191,"children":1193},{"class":702,"line":1192},13,[1194,1199,1203,1208,1212,1216,1220,1224],{"type":22,"tag":700,"props":1195,"children":1196},{"style":707},[1197],{"type":28,"value":1198},"      const",{"type":22,"tag":700,"props":1200,"children":1201},{"style":707},[1202],{"type":28,"value":1050},{"type":22,"tag":700,"props":1204,"children":1205},{"style":719},[1206],{"type":28,"value":1207}," currentTime ",{"type":22,"tag":700,"props":1209,"children":1210},{"style":707},[1211],{"type":28,"value":1060},{"type":22,"tag":700,"props":1213,"children":1214},{"style":713},[1215],{"type":28,"value":970},{"type":22,"tag":700,"props":1217,"children":1218},{"style":719},[1219],{"type":28,"value":910},{"type":22,"tag":700,"props":1221,"children":1222},{"style":713},[1223],{"type":28,"value":1157},{"type":22,"tag":700,"props":1225,"children":1226},{"style":719},[1227],{"type":28,"value":777},{"type":22,"tag":700,"props":1229,"children":1231},{"class":702,"line":1230},14,[1232,1236,1240,1245,1249,1254,1259,1264,1269,1274],{"type":22,"tag":700,"props":1233,"children":1234},{"style":707},[1235],{"type":28,"value":1198},{"type":22,"tag":700,"props":1237,"children":1238},{"style":707},[1239],{"type":28,"value":1050},{"type":22,"tag":700,"props":1241,"children":1242},{"style":719},[1243],{"type":28,"value":1244}," remainingTime ",{"type":22,"tag":700,"props":1246,"children":1247},{"style":707},[1248],{"type":28,"value":1060},{"type":22,"tag":700,"props":1250,"children":1251},{"style":713},[1252],{"type":28,"value":1253}," duration_cast",{"type":22,"tag":700,"props":1255,"children":1256},{"style":719},[1257],{"type":28,"value":1258},"\u003C",{"type":22,"tag":700,"props":1260,"children":1261},{"style":713},[1262],{"type":28,"value":1263},"microseconds",{"type":22,"tag":700,"props":1265,"children":1266},{"style":719},[1267],{"type":28,"value":1268},">(nextTime ",{"type":22,"tag":700,"props":1270,"children":1271},{"style":707},[1272],{"type":28,"value":1273},"-",{"type":22,"tag":700,"props":1275,"children":1276},{"style":719},[1277],{"type":28,"value":1278}," currentTime);\n",{"type":22,"tag":700,"props":1280,"children":1282},{"class":702,"line":1281},15,[1283,1288,1293,1298,1303,1308,1313,1318],{"type":22,"tag":700,"props":1284,"children":1285},{"style":707},[1286],{"type":28,"value":1287},"      if",{"type":22,"tag":700,"props":1289,"children":1290},{"style":719},[1291],{"type":28,"value":1292}," (remainingTime.",{"type":22,"tag":700,"props":1294,"children":1295},{"style":713},[1296],{"type":28,"value":1297},"count",{"type":22,"tag":700,"props":1299,"children":1300},{"style":719},[1301],{"type":28,"value":1302},"() ",{"type":22,"tag":700,"props":1304,"children":1305},{"style":707},[1306],{"type":28,"value":1307},"\u003C=",{"type":22,"tag":700,"props":1309,"children":1310},{"style":746},[1311],{"type":28,"value":1312}," 0",{"type":22,"tag":700,"props":1314,"children":1315},{"style":707},[1316],{"type":28,"value":1317},"LL",{"type":22,"tag":700,"props":1319,"children":1320},{"style":719},[1321],{"type":28,"value":754},{"type":22,"tag":700,"props":1323,"children":1325},{"class":702,"line":1324},16,[1326],{"type":22,"tag":700,"props":1327,"children":1328},{"style":719},[1329],{"type":28,"value":1330},"      {\n",{"type":22,"tag":700,"props":1332,"children":1334},{"class":702,"line":1333},17,[1335,1340],{"type":22,"tag":700,"props":1336,"children":1337},{"style":713},[1338],{"type":28,"value":1339},"         DoMyThing",{"type":22,"tag":700,"props":1341,"children":1342},{"style":719},[1343],{"type":28,"value":777},{"type":22,"tag":700,"props":1345,"children":1347},{"class":702,"line":1346},18,[1348,1353,1357,1361,1366,1370,1374],{"type":22,"tag":700,"props":1349,"children":1350},{"style":719},[1351],{"type":28,"value":1352},"         nextTime ",{"type":22,"tag":700,"props":1354,"children":1355},{"style":707},[1356],{"type":28,"value":1060},{"type":22,"tag":700,"props":1358,"children":1359},{"style":719},[1360],{"type":28,"value":1207},{"type":22,"tag":700,"props":1362,"children":1363},{"style":707},[1364],{"type":28,"value":1365},"+",{"type":22,"tag":700,"props":1367,"children":1368},{"style":719},[1369],{"type":28,"value":1055},{"type":22,"tag":700,"props":1371,"children":1372},{"style":707},[1373],{"type":28,"value":1365},{"type":22,"tag":700,"props":1375,"children":1376},{"style":719},[1377],{"type":28,"value":1378}," remainingTime;\n",{"type":22,"tag":700,"props":1380,"children":1382},{"class":702,"line":1381},19,[1383,1388,1392],{"type":22,"tag":700,"props":1384,"children":1385},{"style":719},[1386],{"type":28,"value":1387},"         sleepTime ",{"type":22,"tag":700,"props":1389,"children":1390},{"style":707},[1391],{"type":28,"value":1060},{"type":22,"tag":700,"props":1393,"children":1394},{"style":719},[1395],{"type":28,"value":1396}," maxSleepTime;\n",{"type":22,"tag":700,"props":1398,"children":1400},{"class":702,"line":1399},20,[1401],{"type":22,"tag":700,"props":1402,"children":1403},{"style":719},[1404],{"type":28,"value":1405},"      }\n",{"type":22,"tag":700,"props":1407,"children":1409},{"class":702,"line":1408},21,[1410],{"type":22,"tag":700,"props":1411,"children":1412},{"style":707},[1413],{"type":28,"value":1414},"      else\n",{"type":22,"tag":700,"props":1416,"children":1418},{"class":702,"line":1417},22,[1419],{"type":22,"tag":700,"props":1420,"children":1421},{"style":719},[1422],{"type":28,"value":1330},{"type":22,"tag":700,"props":1424,"children":1426},{"class":702,"line":1425},23,[1427,1431,1435,1439,1443,1448],{"type":22,"tag":700,"props":1428,"children":1429},{"style":719},[1430],{"type":28,"value":1387},{"type":22,"tag":700,"props":1432,"children":1433},{"style":707},[1434],{"type":28,"value":1060},{"type":22,"tag":700,"props":1436,"children":1437},{"style":713},[1438],{"type":28,"value":905},{"type":22,"tag":700,"props":1440,"children":1441},{"style":719},[1442],{"type":28,"value":910},{"type":22,"tag":700,"props":1444,"children":1445},{"style":713},[1446],{"type":28,"value":1447},"min",{"type":22,"tag":700,"props":1449,"children":1450},{"style":719},[1451],{"type":28,"value":1452},"(maxSleepTime, remainingTime);\n",{"type":22,"tag":700,"props":1454,"children":1456},{"class":702,"line":1455},24,[1457],{"type":22,"tag":700,"props":1458,"children":1459},{"style":719},[1460],{"type":28,"value":1405},{"type":22,"tag":700,"props":1462,"children":1464},{"class":702,"line":1463},25,[1465],{"type":22,"tag":700,"props":1466,"children":1467},{"style":719},[1468],{"type":28,"value":1469},"  \n",{"type":22,"tag":700,"props":1471,"children":1473},{"class":702,"line":1472},26,[1474,1479,1483,1488,1492,1497],{"type":22,"tag":700,"props":1475,"children":1476},{"style":713},[1477],{"type":28,"value":1478},"      std",{"type":22,"tag":700,"props":1480,"children":1481},{"style":719},[1482],{"type":28,"value":910},{"type":22,"tag":700,"props":1484,"children":1485},{"style":713},[1486],{"type":28,"value":1487},"this_thread",{"type":22,"tag":700,"props":1489,"children":1490},{"style":719},[1491],{"type":28,"value":910},{"type":22,"tag":700,"props":1493,"children":1494},{"style":713},[1495],{"type":28,"value":1496},"sleep_for",{"type":22,"tag":700,"props":1498,"children":1499},{"style":719},[1500],{"type":28,"value":1501},"(remainingTime);\n",{"type":22,"tag":700,"props":1503,"children":1505},{"class":702,"line":1504},27,[1506],{"type":22,"tag":700,"props":1507,"children":1508},{"style":719},[1509],{"type":28,"value":816},{"type":22,"tag":700,"props":1511,"children":1513},{"class":702,"line":1512},28,[1514],{"type":22,"tag":700,"props":1515,"children":1516},{"style":719},[1517],{"type":28,"value":825},{"type":22,"tag":96,"props":1519,"children":1520},{},[1521],{"type":22,"tag":31,"props":1522,"children":1523},{},[1524,1529,1531,1536],{"type":22,"tag":41,"props":1525,"children":1526},{},[1527],{"type":28,"value":1528},"Exercise 2",{"type":28,"value":1530},": Now how much time is there between each call to ",{"type":22,"tag":155,"props":1532,"children":1534},{"className":1533},[],[1535],{"type":28,"value":844},{"type":28,"value":846},{"type":22,"tag":31,"props":1538,"children":1539},{},[1540,1542,1548],{"type":28,"value":1541},"Notice that it still isn't enough to just sleep for the interval at which you\nwant to do your thing. Even the higher resolution will have some slop, called\njitter, around its timings. To combat that we do a little polling, waking up to\nsee if it is time then sleeping for the remainder of the time until we do our\nthing. You can try to optimize ",{"type":22,"tag":155,"props":1543,"children":1545},{"className":1544},[],[1546],{"type":28,"value":1547},"kMaxSleepTime",{"type":28,"value":1549}," to improve accuracy and minimize\ntimes being awakened before the work needs to be done.",{"type":22,"tag":31,"props":1551,"children":1552},{},[1553],{"type":28,"value":1554},"You may have noticed this subtlety, it is where some magic happens",{"type":22,"tag":274,"props":1556,"children":1559},{"className":1557,"code":1558,"language":28},[277],"nextTime = currentTime + kMyThingFrequency + remainingTime;\n",[1560],{"type":22,"tag":155,"props":1561,"children":1562},{"__ignoreMap":8},[1563],{"type":28,"value":1558},{"type":22,"tag":31,"props":1565,"children":1566},{},[1567,1569,1575],{"type":28,"value":1568},"The remaining time will be negative or zero when this block executes, so it\nautomatically handles being woken up a little late. Otherwise, ",{"type":22,"tag":155,"props":1570,"children":1572},{"className":1571},[],[1573],{"type":28,"value":1574},"nextTime",{"type":28,"value":1576}," would\naccumulate those errors.",{"type":22,"tag":31,"props":1578,"children":1579},{},[1580,1582,1587,1589,1594,1596,1602,1604,1609],{"type":28,"value":1581},"Another subtlety about ",{"type":22,"tag":155,"props":1583,"children":1585},{"className":1584},[],[1586],{"type":28,"value":1547},{"type":28,"value":1588},". ",{"type":22,"tag":155,"props":1590,"children":1592},{"className":1591},[],[1593],{"type":28,"value":844},{"type":28,"value":1595}," obviously must take less\ntime than ",{"type":22,"tag":155,"props":1597,"children":1599},{"className":1598},[],[1600],{"type":28,"value":1601},"kMyThingFrequency",{"type":28,"value":1603},". How long it takes will influence the value you\nassign to ",{"type":22,"tag":155,"props":1605,"children":1607},{"className":1606},[],[1608],{"type":28,"value":1547},{"type":28,"value":1610}," which needs to be less than the difference between how\nlong your thing takes and how often your thing needs to happen.",{"type":22,"tag":306,"props":1612,"children":1614},{"id":1613},"a-note-on-stdchrono",[1615,1617],{"type":28,"value":1616},"A note on ",{"type":22,"tag":155,"props":1618,"children":1620},{"className":1619},[],[1621],{"type":28,"value":878},{"type":22,"tag":31,"props":1623,"children":1624},{},[1625,1627,1632],{"type":28,"value":1626},"The ",{"type":22,"tag":155,"props":1628,"children":1630},{"className":1629},[],[1631],{"type":28,"value":878},{"type":28,"value":1633}," code can be a little ugly without the aliases. But the unit\nhandling is superb. Once you are used to it you may wish to do similar things\nfor other values with units. It is well worth getting comfortable with it\nwhen you have to do any time based calculations.",{"type":22,"tag":1635,"props":1636,"children":1637},"style",{},[1638],{"type":28,"value":1639},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":8,"searchDepth":387,"depth":387,"links":1641},[1642,1643],{"id":673,"depth":390,"text":676},{"id":854,"depth":390,"text":857,"children":1644},[1645],{"id":1613,"depth":387,"text":1646},"A note on std::chrono","content:jbagley:2021-08-01:AccurateTiming.md","jbagley/2021-08-01/AccurateTiming.md","jbagley/2021-08-01/AccurateTiming",{"user":406,"name":407},{"_path":1652,"_dir":1653,"_draft":7,"_partial":7,"_locale":8,"title":1654,"description":1655,"tags":1656,"excerpt":1655,"image":1659,"publishDate":1660,"body":1661,"_type":399,"_id":1792,"_source":401,"_file":1793,"_stem":1794,"_extension":404,"author":1795},"/jbagley/2021-07/softwaresenescence","2021-07","Legacy Vulnerabilities AKA Software Senescence","Does your business still have an XT computer in the back office because it's\nrunning that one version of some database software that your business depends\non? Yeah, we know there is. Most modern software doesn't work like that.",[1657,1658],"legacy","project-management","/jbagley/2021-07/img/old_software_to_new.jpg","2021-07-01",{"type":19,"children":1662,"toc":1783},[1663,1667,1672,1677,1683,1688,1694,1699,1705,1721,1726,1732,1737,1743,1748,1753,1759,1764,1768,1773,1778],{"type":22,"tag":31,"props":1664,"children":1665},{},[1666],{"type":28,"value":1655},{"type":22,"tag":31,"props":1668,"children":1669},{},[1670],{"type":28,"value":1671},"If you aren't keeping your custom software up with the changing computing\nenvironment, it will fail not necessarily because it has flaws, or the hardware\ncan no longer meet the demand, but because the support your software relies upon\nhas changed.",{"type":22,"tag":31,"props":1673,"children":1674},{},[1675],{"type":28,"value":1676},"Let's look at the vulnerabilities you must manage so that your software does not\nreach its end of life before losing its inherent usefulness.",{"type":22,"tag":59,"props":1678,"children":1680},{"id":1679},"hardware",[1681],{"type":28,"value":1682},"Hardware",{"type":22,"tag":31,"props":1684,"children":1685},{},[1686],{"type":28,"value":1687},"Even though hardware is not changing as frequently as it used to when speed\nimprovements necessitated frequent hardware updates, there is still a little\nrisk. Look at Apple's change to its own processor from Intel's. These changes\ncan be painful, but happily they have slowed to be less than decadal rather than\nyearly.",{"type":22,"tag":59,"props":1689,"children":1691},{"id":1690},"operating-system",[1692],{"type":28,"value":1693},"Operating System",{"type":22,"tag":31,"props":1695,"children":1696},{},[1697],{"type":28,"value":1698},"Operating system vendors have been great accelerators of software development\nthrough frameworks that give your custom software a leg up. Often working\ndirectly with these is necessary to give the kind of experience users\nexpect. And, for some time now, there are also programming languages connected\nto these platforms. Those tied to commercial organizations will be changing\nfrequently due to competitive pressure if nothing else, but open source\nplatforms also need to remain relevant and can introduce changes that break your\nsoftware.",{"type":22,"tag":59,"props":1700,"children":1702},{"id":1701},"third-party-libraries",[1703],{"type":28,"value":1704},"Third Party Libraries",{"type":22,"tag":31,"props":1706,"children":1707},{},[1708,1710,1719],{"type":28,"value":1709},"Whether open source software or commercial, the libraries your software uses\nwill be moving ahead with somewhat less concern for how it affects your software\nthan you have. Even something like a fundamental math library will probably be\nchurning out improvements despite no new numbers having been invented since\n(zero)",{"type":22,"tag":700,"props":1711,"children":1712},{},[1713],{"type":22,"tag":110,"props":1714,"children":1717},{"href":1715,"rel":1716},"https://en.wikipedia.org/wiki/0",[114],[1718],{"type":28,"value":1715},{"type":28,"value":1720}," was invented about 1600 years ago.",{"type":22,"tag":31,"props":1722,"children":1723},{},[1724],{"type":28,"value":1725},"They also can have the same vulnerabilities as your own software and potentially\nmust evolve or fall into irrelevance.",{"type":22,"tag":59,"props":1727,"children":1729},{"id":1728},"protocols-standards-and-formats",[1730],{"type":28,"value":1731},"Protocols, Standards and Formats",{"type":22,"tag":31,"props":1733,"children":1734},{},[1735],{"type":28,"value":1736},"Your software is using common data formats or communicating with other devices\nin some agreed way, a protocol. They may be standardized or may not, but they\nwill often be moving forward in technology's seemingly unstoppable\nmarch. When they change it can ripple out just like an OS update. All the\nsoftware implementing the protocols, standards and formats will need to\nchange, whether yours or third party.",{"type":22,"tag":59,"props":1738,"children":1740},{"id":1739},"remote-interactions",[1741],{"type":28,"value":1742},"Remote Interactions",{"type":22,"tag":31,"props":1744,"children":1745},{},[1746],{"type":28,"value":1747},"Your software sends messages into the ether to get some result from a\nservice. That service is software and has the same vulnerabilities to being left\nbehind as your software. If they evolve your software can lose functionality or\nfail should you not keep up with those changes.",{"type":22,"tag":31,"props":1749,"children":1750},{},[1751],{"type":28,"value":1752},"In addition, nearly unique to this risk, services go away. When they do, it\nleaves you scrambling to find a replacement that minimizes the cost to recover\nthe functionality.",{"type":22,"tag":59,"props":1754,"children":1756},{"id":1755},"security",[1757],{"type":28,"value":1758},"Security",{"type":22,"tag":31,"props":1760,"children":1761},{},[1762],{"type":28,"value":1763},"Underlying all of this is keeping your data and your users safe from\nunauthorized intrusions. It is the most important reason to keep on top of your\ndependencies and so cannot be ignored. Security compounds the above risks as\neach dependency of your software has the same exposure to security risks as your\nown. I put this last in the list to emphasize that it overshadows all the other\nrisks.",{"type":22,"tag":59,"props":1765,"children":1766},{"id":368},[1767],{"type":28,"value":371},{"type":22,"tag":31,"props":1769,"children":1770},{},[1771],{"type":28,"value":1772},"It is often frustrating to incur the cost to make changes because they don't\nresult in new or improved functionality. However, these moments can also be seen\nas opportunities to get in those enhancements you've been planning while\nreducing the overhead of a developer ramping up to being efficient on your\nproject again if it is not in active development.",{"type":22,"tag":31,"props":1774,"children":1775},{},[1776],{"type":28,"value":1777},"Moreso, when you want some feature added and find that your software is so far\nbehind that there is unplanned work to get it up to date and ready for the new\nwork to be done...I imagine that would be most frustrating.",{"type":22,"tag":31,"props":1779,"children":1780},{},[1781],{"type":28,"value":1782},"Once your software falls behind, catching up can be far more costly than it\nwould have been to make the gradual changes needed to prevent the avoidable\nsoftware senescence.",{"title":8,"searchDepth":387,"depth":387,"links":1784},[1785,1786,1787,1788,1789,1790,1791],{"id":1679,"depth":390,"text":1682},{"id":1690,"depth":390,"text":1693},{"id":1701,"depth":390,"text":1704},{"id":1728,"depth":390,"text":1731},{"id":1739,"depth":390,"text":1742},{"id":1755,"depth":390,"text":1758},{"id":368,"depth":390,"text":371},"content:jbagley:2021-07:SoftwareSenescence.md","jbagley/2021-07/SoftwareSenescence.md","jbagley/2021-07/SoftwareSenescence",{"user":406,"name":407},{"_path":1797,"_dir":1798,"_draft":7,"_partial":7,"_locale":8,"title":1799,"description":1800,"tags":1801,"excerpt":1800,"image":1804,"publishDate":1805,"body":1806,"_type":399,"_id":4249,"_source":401,"_file":4250,"_stem":4251,"_extension":404,"author":4252},"/jbagley/2019-4/makingspectrogramsinjuce","2019-4","Making Spectrograms in JUCE","Art+Logic's Incubator project has made a lot of progress. In a previous post I mentioned that Dr. Scott Hawley's technique to classify audio involved converting audio to an image and using a Convolution Neural Network (CNN) to classify the audio based on this image. That image is a spectrogram. I'm going to go into some detail about what we do to create one, and why to the best of my ability.",[1802,662,1803],"juce","audio","/jbagley/2019-4/img/Fortissimo_Trumpet_Ensemble_Matrix_Swells_61.wav-2048x1700.png","2019-04-01",{"type":19,"children":1807,"toc":4239},[1808,1822,1828,1849,1855,1867,1880,1901,1920,1925,2100,2106,2118,2791,2797,2802,2838,3089,3094,3360,3380,3385,3391,3405,3419,3472,3478,3498,3914,3920,3940,4188,4194,4199,4204,4235],{"type":22,"tag":31,"props":1809,"children":1810},{},[1811,1813,1820],{"type":28,"value":1812},"Art+Logic's Incubator project has made a lot of progress. In a ",{"type":22,"tag":110,"props":1814,"children":1817},{"href":1815,"rel":1816},"https://artandlogic.com/2018/12/incubator-kick-off/",[114],[1818],{"type":28,"value":1819},"previous post",{"type":28,"value":1821}," I mentioned that Dr. Scott Hawley's technique to classify audio involved converting audio to an image and using a Convolution Neural Network (CNN) to classify the audio based on this image. That image is a spectrogram. I'm going to go into some detail about what we do to create one, and why to the best of my ability.",{"type":22,"tag":59,"props":1823,"children":1825},{"id":1824},"basics",[1826],{"type":28,"value":1827},"Basics",{"type":22,"tag":31,"props":1829,"children":1830},{},[1831,1833,1839,1841,1847],{"type":28,"value":1832},"The spectrogram shows the energy within a set of frequency ranges. The vertical axis represents the frequencies, the horizontal axis represents time. When represented as a grayscale image, the brighter the pixel, the more energy in that frequency range (",{"type":22,"tag":155,"props":1834,"children":1836},{"className":1835},[],[1837],{"type":28,"value":1838},"y",{"type":28,"value":1840},") at that time range (",{"type":22,"tag":155,"props":1842,"children":1844},{"className":1843},[],[1845],{"type":28,"value":1846},"x",{"type":28,"value":1848},").",{"type":22,"tag":59,"props":1850,"children":1852},{"id":1851},"the-base-the-fft",[1853],{"type":28,"value":1854},"The Base -- The FFT",{"type":22,"tag":31,"props":1856,"children":1857},{},[1858,1860,1865],{"type":28,"value":1859},"To create the spectrogram we have to extract that frequency data from the audio signal. This is done with a Fourier Transform algorithm, commonly implemented as the Fast Fourier Transform (FFT). The variation we need to use is the Short Time Fourier Transform (STFT) so we can see how the frequency information changes over the time. In general, a Fourier Transform takes a signal, for us a buffer full of discrete samples taken of a signal over time (i.e. audio), isolates contiguous frequency ranges into what are called bins, then measures the energy present in each bin over some length of time. More on ",{"type":22,"tag":700,"props":1861,"children":1862},{},[1863],{"type":28,"value":1864},"STFT",{"type":28,"value":1866}," later.",{"type":22,"tag":31,"props":1868,"children":1869},{},[1870,1872,1878],{"type":28,"value":1871},"JUCE provides an implementation of an FFT that we can use to implement the STFT in its ",{"type":22,"tag":155,"props":1873,"children":1875},{"className":1874},[],[1876],{"type":28,"value":1877},"dsp",{"type":28,"value":1879}," module which it documents as not necessarily fast but good enough for many uses. So far it works for us. We may be lucky in that we've been developing on macOS, where JUCE uses Apple's Accelerate library.",{"type":22,"tag":31,"props":1881,"children":1882},{},[1883,1885,1891,1893,1899],{"type":28,"value":1884},"The FFT class is initialized with an ",{"type":22,"tag":155,"props":1886,"children":1888},{"className":1887},[],[1889],{"type":28,"value":1890},"order",{"type":28,"value":1892}," argument. The order is the exponent such that the number of samples processed by the FFT will be two raised to that number, called ",{"type":22,"tag":155,"props":1894,"children":1896},{"className":1895},[],[1897],{"type":28,"value":1898},"fftSize",{"type":28,"value":1900},". It wasn't immediately obvious to me what the size was when I started on this code. Is it the number of bins, or is it the number of samples the FFT will produce? For JUCE and Accelerate it's both.",{"type":22,"tag":31,"props":1902,"children":1903},{},[1904,1906,1911,1913,1918],{"type":28,"value":1905},"I thought I would be able to simply pass it the number of bins I wanted, and define the length separately. Using ",{"type":22,"tag":155,"props":1907,"children":1909},{"className":1908},[],[1910],{"type":28,"value":1890},{"type":28,"value":1912}," this way makes sense when you consider that the FFT works best when the number of samples it will process is a power of two. Additionally, the Inverse Fourier Transform can be performed, in which case the input is the bins and there would need to be ",{"type":22,"tag":155,"props":1914,"children":1916},{"className":1915},[],[1917],{"type":28,"value":1898},{"type":28,"value":1919}," bins.",{"type":22,"tag":31,"props":1921,"children":1922},{},[1923],{"type":28,"value":1924},"The total number of bins, in our case 2048, includes positive and negative frequency values. The two halves will be mirrored values, and we'll eventually only keep the positive half of them.",{"type":22,"tag":274,"props":1926,"children":1928},{"className":692,"code":1927,"language":694,"meta":8,"style":8},"/// The class to make spectrograms from audio file data\nclass ASpectroMaker\n{\n   //...\n   dsp::FFT fFft;\n   //...\n};\n\nASpectroMaker::ASpectroMaker(int fftBins /*...*/)\n: fFft(std::log2(fftBins))\n// ...\n{\n}\n",[1929],{"type":22,"tag":155,"props":1930,"children":1931},{"__ignoreMap":8},[1932,1940,1953,1960,1968,1981,1988,1996,2003,2042,2078,2086,2093],{"type":22,"tag":700,"props":1933,"children":1934},{"class":702,"line":703},[1935],{"type":22,"tag":700,"props":1936,"children":1937},{"style":804},[1938],{"type":28,"value":1939},"/// The class to make spectrograms from audio file data\n",{"type":22,"tag":700,"props":1941,"children":1942},{"class":702,"line":390},[1943,1948],{"type":22,"tag":700,"props":1944,"children":1945},{"style":707},[1946],{"type":28,"value":1947},"class",{"type":22,"tag":700,"props":1949,"children":1950},{"style":713},[1951],{"type":28,"value":1952}," ASpectroMaker\n",{"type":22,"tag":700,"props":1954,"children":1955},{"class":702,"line":387},[1956],{"type":22,"tag":700,"props":1957,"children":1958},{"style":719},[1959],{"type":28,"value":730},{"type":22,"tag":700,"props":1961,"children":1962},{"class":702,"line":757},[1963],{"type":22,"tag":700,"props":1964,"children":1965},{"style":804},[1966],{"type":28,"value":1967},"   //...\n",{"type":22,"tag":700,"props":1969,"children":1970},{"class":702,"line":766},[1971,1976],{"type":22,"tag":700,"props":1972,"children":1973},{"style":713},[1974],{"type":28,"value":1975},"   dsp",{"type":22,"tag":700,"props":1977,"children":1978},{"style":719},[1979],{"type":28,"value":1980},"::FFT fFft;\n",{"type":22,"tag":700,"props":1982,"children":1983},{"class":702,"line":780},[1984],{"type":22,"tag":700,"props":1985,"children":1986},{"style":804},[1987],{"type":28,"value":1967},{"type":22,"tag":700,"props":1989,"children":1990},{"class":702,"line":810},[1991],{"type":22,"tag":700,"props":1992,"children":1993},{"style":719},[1994],{"type":28,"value":1995},"};\n",{"type":22,"tag":700,"props":1997,"children":1998},{"class":702,"line":819},[1999],{"type":22,"tag":700,"props":2000,"children":2001},{"emptyLinePlaceholder":1011},[2002],{"type":28,"value":1014},{"type":22,"tag":700,"props":2004,"children":2005},{"class":702,"line":1121},[2006,2011,2015,2019,2023,2028,2033,2038],{"type":22,"tag":700,"props":2007,"children":2008},{"style":713},[2009],{"type":28,"value":2010},"ASpectroMaker",{"type":22,"tag":700,"props":2012,"children":2013},{"style":719},[2014],{"type":28,"value":910},{"type":22,"tag":700,"props":2016,"children":2017},{"style":713},[2018],{"type":28,"value":2010},{"type":22,"tag":700,"props":2020,"children":2021},{"style":719},[2022],{"type":28,"value":791},{"type":22,"tag":700,"props":2024,"children":2025},{"style":707},[2026],{"type":28,"value":2027},"int",{"type":22,"tag":700,"props":2029,"children":2030},{"style":719},[2031],{"type":28,"value":2032}," fftBins",{"type":22,"tag":700,"props":2034,"children":2035},{"style":804},[2036],{"type":28,"value":2037}," /*...*/",{"type":22,"tag":700,"props":2039,"children":2040},{"style":719},[2041],{"type":28,"value":754},{"type":22,"tag":700,"props":2043,"children":2044},{"class":702,"line":1129},[2045,2050,2055,2059,2064,2068,2073],{"type":22,"tag":700,"props":2046,"children":2047},{"style":719},[2048],{"type":28,"value":2049},": ",{"type":22,"tag":700,"props":2051,"children":2052},{"style":713},[2053],{"type":28,"value":2054},"fFft",{"type":22,"tag":700,"props":2056,"children":2057},{"style":719},[2058],{"type":28,"value":791},{"type":22,"tag":700,"props":2060,"children":2061},{"style":713},[2062],{"type":28,"value":2063},"std",{"type":22,"tag":700,"props":2065,"children":2066},{"style":719},[2067],{"type":28,"value":910},{"type":22,"tag":700,"props":2069,"children":2070},{"style":713},[2071],{"type":28,"value":2072},"log2",{"type":22,"tag":700,"props":2074,"children":2075},{"style":719},[2076],{"type":28,"value":2077},"(fftBins))\n",{"type":22,"tag":700,"props":2079,"children":2080},{"class":702,"line":1164},[2081],{"type":22,"tag":700,"props":2082,"children":2083},{"style":804},[2084],{"type":28,"value":2085},"// ...\n",{"type":22,"tag":700,"props":2087,"children":2088},{"class":702,"line":1184},[2089],{"type":22,"tag":700,"props":2090,"children":2091},{"style":719},[2092],{"type":28,"value":730},{"type":22,"tag":700,"props":2094,"children":2095},{"class":702,"line":1192},[2096],{"type":22,"tag":700,"props":2097,"children":2098},{"style":719},[2099],{"type":28,"value":825},{"type":22,"tag":59,"props":2101,"children":2103},{"id":2102},"the-iteration-the-stft",[2104],{"type":28,"value":2105},"The Iteration -- The STFT",{"type":22,"tag":31,"props":2107,"children":2108},{},[2109,2111,2116],{"type":28,"value":2110},"In STFT The ",{"type":22,"tag":37,"props":2112,"children":2113},{},[2114],{"type":28,"value":2115},"Short Time",{"type":28,"value":2117}," part means we will take overlapping subsets of the signal and perform the transform on each chunk. So each column of the spectrogram represents the Fourier transform output from one of these chunks. The distance between chunks is called the hop size. The hop size should be less than the chunk size.",{"type":22,"tag":274,"props":2119,"children":2121},{"className":692,"code":2120,"language":694,"meta":8,"style":8},"using Spectrum = std::vector\u003Cfloat>; //!\u003C One audio channel of FFT data over time, really 2-dimensional\n\nASpectrogram ASpectroMaker::STFT(const AudioSampleBuffer& signal, size_t hop)\n{\n   const float* data = signal.getReadPointer(0);\n   const size_t dataCount = signal.getNumSamples();\n\n   // fftSize will be the number of bins we used to initialize the ASpectroMaker.\n   ptrdiff_t fftSize = fFft.getSize();\n\n   // See below for more about numHops\n   ptrdiff_t numHops = 1L + static_cast\u003Clong>((dataCount - fftSize) / hop);\n\n   //...\n\n   // Ignore the negative frequency information but leave the center bin.\n   size_t numRows = 1UL + (fftSize / 2UL);\n\n   // fFft works on the data in place, and needs twice as much space as the input size.\n   std::vector\u003Cfloat> fftBuffer(fftSize * 2UL);\n\n   // While data remains\n   {\n      std::memcpy(fftBuffer.data(), data, fftSize * sizeof(float));\n\n      // prepare fft data...\n\n      fFft.performFrequencyOnlyForwardTransform(fftBuffer.data());\n\n      // ...copy the frequency information from fftBuffer to the spectrum\n\n      // Next chunk\n      data += hop;\n   }\n   //...\n}\n",[2122],{"type":22,"tag":155,"props":2123,"children":2124},{"__ignoreMap":8},[2125,2173,2180,2245,2252,2297,2331,2338,2346,2377,2384,2392,2450,2457,2464,2471,2479,2531,2538,2546,2591,2598,2606,2613,2666,2673,2681,2688,2714,2722,2731,2739,2748,2767,2775,2783],{"type":22,"tag":700,"props":2126,"children":2127},{"class":702,"line":703},[2128,2132,2137,2141,2145,2149,2154,2158,2163,2168],{"type":22,"tag":700,"props":2129,"children":2130},{"style":707},[2131],{"type":28,"value":895},{"type":22,"tag":700,"props":2133,"children":2134},{"style":713},[2135],{"type":28,"value":2136}," Spectrum",{"type":22,"tag":700,"props":2138,"children":2139},{"style":707},[2140],{"type":28,"value":975},{"type":22,"tag":700,"props":2142,"children":2143},{"style":713},[2144],{"type":28,"value":905},{"type":22,"tag":700,"props":2146,"children":2147},{"style":719},[2148],{"type":28,"value":910},{"type":22,"tag":700,"props":2150,"children":2151},{"style":713},[2152],{"type":28,"value":2153},"vector",{"type":22,"tag":700,"props":2155,"children":2156},{"style":719},[2157],{"type":28,"value":1258},{"type":22,"tag":700,"props":2159,"children":2160},{"style":707},[2161],{"type":28,"value":2162},"float",{"type":22,"tag":700,"props":2164,"children":2165},{"style":719},[2166],{"type":28,"value":2167},">;",{"type":22,"tag":700,"props":2169,"children":2170},{"style":804},[2171],{"type":28,"value":2172}," //!\u003C One audio channel of FFT data over time, really 2-dimensional\n",{"type":22,"tag":700,"props":2174,"children":2175},{"class":702,"line":390},[2176],{"type":22,"tag":700,"props":2177,"children":2178},{"emptyLinePlaceholder":1011},[2179],{"type":28,"value":1014},{"type":22,"tag":700,"props":2181,"children":2182},{"class":702,"line":387},[2183,2188,2193,2197,2201,2205,2210,2215,2220,2226,2231,2236,2241],{"type":22,"tag":700,"props":2184,"children":2185},{"style":713},[2186],{"type":28,"value":2187},"ASpectrogram",{"type":22,"tag":700,"props":2189,"children":2190},{"style":713},[2191],{"type":28,"value":2192}," ASpectroMaker",{"type":22,"tag":700,"props":2194,"children":2195},{"style":719},[2196],{"type":28,"value":910},{"type":22,"tag":700,"props":2198,"children":2199},{"style":713},[2200],{"type":28,"value":1864},{"type":22,"tag":700,"props":2202,"children":2203},{"style":719},[2204],{"type":28,"value":791},{"type":22,"tag":700,"props":2206,"children":2207},{"style":707},[2208],{"type":28,"value":2209},"const",{"type":22,"tag":700,"props":2211,"children":2212},{"style":713},[2213],{"type":28,"value":2214}," AudioSampleBuffer",{"type":22,"tag":700,"props":2216,"children":2217},{"style":707},[2218],{"type":28,"value":2219},"&",{"type":22,"tag":700,"props":2221,"children":2223},{"style":2222},"--shiki-default:#E36209;--shiki-dark:#FFAB70",[2224],{"type":28,"value":2225}," signal",{"type":22,"tag":700,"props":2227,"children":2228},{"style":719},[2229],{"type":28,"value":2230},", ",{"type":22,"tag":700,"props":2232,"children":2233},{"style":707},[2234],{"type":28,"value":2235},"size_t",{"type":22,"tag":700,"props":2237,"children":2238},{"style":2222},[2239],{"type":28,"value":2240}," hop",{"type":22,"tag":700,"props":2242,"children":2243},{"style":719},[2244],{"type":28,"value":754},{"type":22,"tag":700,"props":2246,"children":2247},{"class":702,"line":757},[2248],{"type":22,"tag":700,"props":2249,"children":2250},{"style":719},[2251],{"type":28,"value":730},{"type":22,"tag":700,"props":2253,"children":2254},{"class":702,"line":766},[2255,2259,2264,2269,2273,2278,2283,2287,2292],{"type":22,"tag":700,"props":2256,"children":2257},{"style":707},[2258],{"type":28,"value":1045},{"type":22,"tag":700,"props":2260,"children":2261},{"style":707},[2262],{"type":28,"value":2263}," float*",{"type":22,"tag":700,"props":2265,"children":2266},{"style":719},[2267],{"type":28,"value":2268}," data ",{"type":22,"tag":700,"props":2270,"children":2271},{"style":707},[2272],{"type":28,"value":1060},{"type":22,"tag":700,"props":2274,"children":2275},{"style":719},[2276],{"type":28,"value":2277}," signal.",{"type":22,"tag":700,"props":2279,"children":2280},{"style":713},[2281],{"type":28,"value":2282},"getReadPointer",{"type":22,"tag":700,"props":2284,"children":2285},{"style":719},[2286],{"type":28,"value":791},{"type":22,"tag":700,"props":2288,"children":2289},{"style":746},[2290],{"type":28,"value":2291},"0",{"type":22,"tag":700,"props":2293,"children":2294},{"style":719},[2295],{"type":28,"value":2296},");\n",{"type":22,"tag":700,"props":2298,"children":2299},{"class":702,"line":780},[2300,2304,2309,2314,2318,2322,2327],{"type":22,"tag":700,"props":2301,"children":2302},{"style":707},[2303],{"type":28,"value":1045},{"type":22,"tag":700,"props":2305,"children":2306},{"style":707},[2307],{"type":28,"value":2308}," size_t",{"type":22,"tag":700,"props":2310,"children":2311},{"style":719},[2312],{"type":28,"value":2313}," dataCount ",{"type":22,"tag":700,"props":2315,"children":2316},{"style":707},[2317],{"type":28,"value":1060},{"type":22,"tag":700,"props":2319,"children":2320},{"style":719},[2321],{"type":28,"value":2277},{"type":22,"tag":700,"props":2323,"children":2324},{"style":713},[2325],{"type":28,"value":2326},"getNumSamples",{"type":22,"tag":700,"props":2328,"children":2329},{"style":719},[2330],{"type":28,"value":777},{"type":22,"tag":700,"props":2332,"children":2333},{"class":702,"line":810},[2334],{"type":22,"tag":700,"props":2335,"children":2336},{"emptyLinePlaceholder":1011},[2337],{"type":28,"value":1014},{"type":22,"tag":700,"props":2339,"children":2340},{"class":702,"line":819},[2341],{"type":22,"tag":700,"props":2342,"children":2343},{"style":804},[2344],{"type":28,"value":2345},"   // fftSize will be the number of bins we used to initialize the ASpectroMaker.\n",{"type":22,"tag":700,"props":2347,"children":2348},{"class":702,"line":1121},[2349,2354,2359,2363,2368,2373],{"type":22,"tag":700,"props":2350,"children":2351},{"style":746},[2352],{"type":28,"value":2353},"   ptrdiff_t",{"type":22,"tag":700,"props":2355,"children":2356},{"style":719},[2357],{"type":28,"value":2358}," fftSize ",{"type":22,"tag":700,"props":2360,"children":2361},{"style":707},[2362],{"type":28,"value":1060},{"type":22,"tag":700,"props":2364,"children":2365},{"style":719},[2366],{"type":28,"value":2367}," fFft.",{"type":22,"tag":700,"props":2369,"children":2370},{"style":713},[2371],{"type":28,"value":2372},"getSize",{"type":22,"tag":700,"props":2374,"children":2375},{"style":719},[2376],{"type":28,"value":777},{"type":22,"tag":700,"props":2378,"children":2379},{"class":702,"line":1129},[2380],{"type":22,"tag":700,"props":2381,"children":2382},{"emptyLinePlaceholder":1011},[2383],{"type":28,"value":1014},{"type":22,"tag":700,"props":2385,"children":2386},{"class":702,"line":1164},[2387],{"type":22,"tag":700,"props":2388,"children":2389},{"style":804},[2390],{"type":28,"value":2391},"   // See below for more about numHops\n",{"type":22,"tag":700,"props":2393,"children":2394},{"class":702,"line":1184},[2395,2399,2404,2408,2412,2417,2422,2427,2432,2436,2441,2445],{"type":22,"tag":700,"props":2396,"children":2397},{"style":746},[2398],{"type":28,"value":2353},{"type":22,"tag":700,"props":2400,"children":2401},{"style":719},[2402],{"type":28,"value":2403}," numHops ",{"type":22,"tag":700,"props":2405,"children":2406},{"style":707},[2407],{"type":28,"value":1060},{"type":22,"tag":700,"props":2409,"children":2410},{"style":746},[2411],{"type":28,"value":1065},{"type":22,"tag":700,"props":2413,"children":2414},{"style":707},[2415],{"type":28,"value":2416},"L",{"type":22,"tag":700,"props":2418,"children":2419},{"style":707},[2420],{"type":28,"value":2421}," +",{"type":22,"tag":700,"props":2423,"children":2424},{"style":707},[2425],{"type":28,"value":2426}," static_cast\u003Clong>",{"type":22,"tag":700,"props":2428,"children":2429},{"style":719},[2430],{"type":28,"value":2431},"((dataCount ",{"type":22,"tag":700,"props":2433,"children":2434},{"style":707},[2435],{"type":28,"value":1273},{"type":22,"tag":700,"props":2437,"children":2438},{"style":719},[2439],{"type":28,"value":2440}," fftSize) ",{"type":22,"tag":700,"props":2442,"children":2443},{"style":707},[2444],{"type":28,"value":1104},{"type":22,"tag":700,"props":2446,"children":2447},{"style":719},[2448],{"type":28,"value":2449}," hop);\n",{"type":22,"tag":700,"props":2451,"children":2452},{"class":702,"line":1192},[2453],{"type":22,"tag":700,"props":2454,"children":2455},{"emptyLinePlaceholder":1011},[2456],{"type":28,"value":1014},{"type":22,"tag":700,"props":2458,"children":2459},{"class":702,"line":1230},[2460],{"type":22,"tag":700,"props":2461,"children":2462},{"style":804},[2463],{"type":28,"value":1967},{"type":22,"tag":700,"props":2465,"children":2466},{"class":702,"line":1281},[2467],{"type":22,"tag":700,"props":2468,"children":2469},{"emptyLinePlaceholder":1011},[2470],{"type":28,"value":1014},{"type":22,"tag":700,"props":2472,"children":2473},{"class":702,"line":1324},[2474],{"type":22,"tag":700,"props":2475,"children":2476},{"style":804},[2477],{"type":28,"value":2478},"   // Ignore the negative frequency information but leave the center bin.\n",{"type":22,"tag":700,"props":2480,"children":2481},{"class":702,"line":1333},[2482,2487,2492,2496,2500,2505,2509,2514,2518,2523,2527],{"type":22,"tag":700,"props":2483,"children":2484},{"style":707},[2485],{"type":28,"value":2486},"   size_t",{"type":22,"tag":700,"props":2488,"children":2489},{"style":719},[2490],{"type":28,"value":2491}," numRows ",{"type":22,"tag":700,"props":2493,"children":2494},{"style":707},[2495],{"type":28,"value":1060},{"type":22,"tag":700,"props":2497,"children":2498},{"style":746},[2499],{"type":28,"value":1065},{"type":22,"tag":700,"props":2501,"children":2502},{"style":707},[2503],{"type":28,"value":2504},"UL",{"type":22,"tag":700,"props":2506,"children":2507},{"style":707},[2508],{"type":28,"value":2421},{"type":22,"tag":700,"props":2510,"children":2511},{"style":719},[2512],{"type":28,"value":2513}," (fftSize ",{"type":22,"tag":700,"props":2515,"children":2516},{"style":707},[2517],{"type":28,"value":1104},{"type":22,"tag":700,"props":2519,"children":2520},{"style":746},[2521],{"type":28,"value":2522}," 2",{"type":22,"tag":700,"props":2524,"children":2525},{"style":707},[2526],{"type":28,"value":2504},{"type":22,"tag":700,"props":2528,"children":2529},{"style":719},[2530],{"type":28,"value":2296},{"type":22,"tag":700,"props":2532,"children":2533},{"class":702,"line":1346},[2534],{"type":22,"tag":700,"props":2535,"children":2536},{"emptyLinePlaceholder":1011},[2537],{"type":28,"value":1014},{"type":22,"tag":700,"props":2539,"children":2540},{"class":702,"line":1381},[2541],{"type":22,"tag":700,"props":2542,"children":2543},{"style":804},[2544],{"type":28,"value":2545},"   // fFft works on the data in place, and needs twice as much space as the input size.\n",{"type":22,"tag":700,"props":2547,"children":2548},{"class":702,"line":1399},[2549,2554,2559,2564,2569,2574,2579,2583,2587],{"type":22,"tag":700,"props":2550,"children":2551},{"style":713},[2552],{"type":28,"value":2553},"   std",{"type":22,"tag":700,"props":2555,"children":2556},{"style":719},[2557],{"type":28,"value":2558},"::vector",{"type":22,"tag":700,"props":2560,"children":2561},{"style":707},[2562],{"type":28,"value":2563},"\u003Cfloat>",{"type":22,"tag":700,"props":2565,"children":2566},{"style":713},[2567],{"type":28,"value":2568}," fftBuffer",{"type":22,"tag":700,"props":2570,"children":2571},{"style":719},[2572],{"type":28,"value":2573},"(fftSize ",{"type":22,"tag":700,"props":2575,"children":2576},{"style":707},[2577],{"type":28,"value":2578},"*",{"type":22,"tag":700,"props":2580,"children":2581},{"style":746},[2582],{"type":28,"value":2522},{"type":22,"tag":700,"props":2584,"children":2585},{"style":707},[2586],{"type":28,"value":2504},{"type":22,"tag":700,"props":2588,"children":2589},{"style":719},[2590],{"type":28,"value":2296},{"type":22,"tag":700,"props":2592,"children":2593},{"class":702,"line":1408},[2594],{"type":22,"tag":700,"props":2595,"children":2596},{"emptyLinePlaceholder":1011},[2597],{"type":28,"value":1014},{"type":22,"tag":700,"props":2599,"children":2600},{"class":702,"line":1417},[2601],{"type":22,"tag":700,"props":2602,"children":2603},{"style":804},[2604],{"type":28,"value":2605},"   // While data remains\n",{"type":22,"tag":700,"props":2607,"children":2608},{"class":702,"line":1425},[2609],{"type":22,"tag":700,"props":2610,"children":2611},{"style":719},[2612],{"type":28,"value":763},{"type":22,"tag":700,"props":2614,"children":2615},{"class":702,"line":1455},[2616,2620,2624,2629,2634,2639,2644,2648,2653,2657,2661],{"type":22,"tag":700,"props":2617,"children":2618},{"style":713},[2619],{"type":28,"value":1478},{"type":22,"tag":700,"props":2621,"children":2622},{"style":719},[2623],{"type":28,"value":910},{"type":22,"tag":700,"props":2625,"children":2626},{"style":713},[2627],{"type":28,"value":2628},"memcpy",{"type":22,"tag":700,"props":2630,"children":2631},{"style":719},[2632],{"type":28,"value":2633},"(fftBuffer.",{"type":22,"tag":700,"props":2635,"children":2636},{"style":713},[2637],{"type":28,"value":2638},"data",{"type":22,"tag":700,"props":2640,"children":2641},{"style":719},[2642],{"type":28,"value":2643},"(), data, fftSize ",{"type":22,"tag":700,"props":2645,"children":2646},{"style":707},[2647],{"type":28,"value":2578},{"type":22,"tag":700,"props":2649,"children":2650},{"style":707},[2651],{"type":28,"value":2652}," sizeof",{"type":22,"tag":700,"props":2654,"children":2655},{"style":719},[2656],{"type":28,"value":791},{"type":22,"tag":700,"props":2658,"children":2659},{"style":707},[2660],{"type":28,"value":2162},{"type":22,"tag":700,"props":2662,"children":2663},{"style":719},[2664],{"type":28,"value":2665},"));\n",{"type":22,"tag":700,"props":2667,"children":2668},{"class":702,"line":1463},[2669],{"type":22,"tag":700,"props":2670,"children":2671},{"emptyLinePlaceholder":1011},[2672],{"type":28,"value":1014},{"type":22,"tag":700,"props":2674,"children":2675},{"class":702,"line":1472},[2676],{"type":22,"tag":700,"props":2677,"children":2678},{"style":804},[2679],{"type":28,"value":2680},"      // prepare fft data...\n",{"type":22,"tag":700,"props":2682,"children":2683},{"class":702,"line":1504},[2684],{"type":22,"tag":700,"props":2685,"children":2686},{"emptyLinePlaceholder":1011},[2687],{"type":28,"value":1014},{"type":22,"tag":700,"props":2689,"children":2690},{"class":702,"line":1512},[2691,2696,2701,2705,2709],{"type":22,"tag":700,"props":2692,"children":2693},{"style":719},[2694],{"type":28,"value":2695},"      fFft.",{"type":22,"tag":700,"props":2697,"children":2698},{"style":713},[2699],{"type":28,"value":2700},"performFrequencyOnlyForwardTransform",{"type":22,"tag":700,"props":2702,"children":2703},{"style":719},[2704],{"type":28,"value":2633},{"type":22,"tag":700,"props":2706,"children":2707},{"style":713},[2708],{"type":28,"value":2638},{"type":22,"tag":700,"props":2710,"children":2711},{"style":719},[2712],{"type":28,"value":2713},"());\n",{"type":22,"tag":700,"props":2715,"children":2717},{"class":702,"line":2716},29,[2718],{"type":22,"tag":700,"props":2719,"children":2720},{"emptyLinePlaceholder":1011},[2721],{"type":28,"value":1014},{"type":22,"tag":700,"props":2723,"children":2725},{"class":702,"line":2724},30,[2726],{"type":22,"tag":700,"props":2727,"children":2728},{"style":804},[2729],{"type":28,"value":2730},"      // ...copy the frequency information from fftBuffer to the spectrum\n",{"type":22,"tag":700,"props":2732,"children":2734},{"class":702,"line":2733},31,[2735],{"type":22,"tag":700,"props":2736,"children":2737},{"emptyLinePlaceholder":1011},[2738],{"type":28,"value":1014},{"type":22,"tag":700,"props":2740,"children":2742},{"class":702,"line":2741},32,[2743],{"type":22,"tag":700,"props":2744,"children":2745},{"style":804},[2746],{"type":28,"value":2747},"      // Next chunk\n",{"type":22,"tag":700,"props":2749,"children":2751},{"class":702,"line":2750},33,[2752,2757,2762],{"type":22,"tag":700,"props":2753,"children":2754},{"style":719},[2755],{"type":28,"value":2756},"      data ",{"type":22,"tag":700,"props":2758,"children":2759},{"style":707},[2760],{"type":28,"value":2761},"+=",{"type":22,"tag":700,"props":2763,"children":2764},{"style":719},[2765],{"type":28,"value":2766}," hop;\n",{"type":22,"tag":700,"props":2768,"children":2770},{"class":702,"line":2769},34,[2771],{"type":22,"tag":700,"props":2772,"children":2773},{"style":719},[2774],{"type":28,"value":816},{"type":22,"tag":700,"props":2776,"children":2778},{"class":702,"line":2777},35,[2779],{"type":22,"tag":700,"props":2780,"children":2781},{"style":804},[2782],{"type":28,"value":1967},{"type":22,"tag":700,"props":2784,"children":2786},{"class":702,"line":2785},36,[2787],{"type":22,"tag":700,"props":2788,"children":2789},{"style":719},[2790],{"type":28,"value":825},{"type":22,"tag":59,"props":2792,"children":2794},{"id":2793},"windowing",[2795],{"type":28,"value":2796},"Windowing",{"type":22,"tag":31,"props":2798,"children":2799},{},[2800],{"type":28,"value":2801},"Because we are breaking apart our signal into chunks, the first and last values in each buffer will probably not be zero. When that's true, the FFT will produce noise in the output due to the discontinuity of the signal. Imagine a flat signal with just zero before and after the chunk. We need to smooth the begining and end to make sure they transition smoothly to the full amplitude of the signal. To do this we use a windowing function. There are many choices for windowing functions with various tradeoffs, and JUCE provides several options.",{"type":22,"tag":31,"props":2803,"children":2804},{},[2805,2806,2813,2815,2821,2823,2828,2830,2836],{"type":28,"value":1626},{"type":22,"tag":110,"props":2807,"children":2810},{"href":2808,"rel":2809},"https://en.wikipedia.org/wiki/Hann_function",[114],[2811],{"type":28,"value":2812},"Hann function",{"type":28,"value":2814}," provides good performance and meets our requirements (and it was what Dr. Hawley used). Notice that we initialize it with the ",{"type":22,"tag":155,"props":2816,"children":2818},{"className":2817},[],[2819],{"type":28,"value":2820},"fftSize + 1",{"type":28,"value":2822},", not the hop size. This threw me for a minute during development, but because we are windowing we have to overlap the chunks, moving by hop size, but processing ",{"type":22,"tag":155,"props":2824,"children":2826},{"className":2825},[],[2827],{"type":28,"value":1898},{"type":28,"value":2829}," samples per chunk, where ",{"type":22,"tag":155,"props":2831,"children":2833},{"className":2832},[],[2834],{"type":28,"value":2835},"hop \u003C fftSize",{"type":28,"value":2837},". Looking at the Hann function it only allows the maximum signal amplitude near the center, the rest of the time it attenuates the signal. So to represent the original signal more accurately we overlap the chunks so that at some iteration the full amplitude will be well represented and in others it still contributes some to the calculations.",{"type":22,"tag":274,"props":2839,"children":2841},{"className":692,"code":2840,"language":694,"meta":8,"style":8},"/// The class to make spectrograms from audio file data\nclass ASpectroMaker\n{\n   //...\n   dsp::FFT fFft;\n   dsp::WindowingFunction\u003Cfloat> fWindow;\n   //...\n};\n\nASpectroMaker::ASpectroMaker(int fftBins /*...*/)\n: fFft(std::log2(fftBins))\n, fWindow(fFft.getSize() + 1, dsp::WindowingFunction\u003Cfloat>::hann, false)\n// ...\n{\n}\n",[2842],{"type":22,"tag":155,"props":2843,"children":2844},{"__ignoreMap":8},[2845,2852,2863,2870,2877,2888,2909,2916,2923,2930,2965,2996,3068,3075,3082],{"type":22,"tag":700,"props":2846,"children":2847},{"class":702,"line":703},[2848],{"type":22,"tag":700,"props":2849,"children":2850},{"style":804},[2851],{"type":28,"value":1939},{"type":22,"tag":700,"props":2853,"children":2854},{"class":702,"line":390},[2855,2859],{"type":22,"tag":700,"props":2856,"children":2857},{"style":707},[2858],{"type":28,"value":1947},{"type":22,"tag":700,"props":2860,"children":2861},{"style":713},[2862],{"type":28,"value":1952},{"type":22,"tag":700,"props":2864,"children":2865},{"class":702,"line":387},[2866],{"type":22,"tag":700,"props":2867,"children":2868},{"style":719},[2869],{"type":28,"value":730},{"type":22,"tag":700,"props":2871,"children":2872},{"class":702,"line":757},[2873],{"type":22,"tag":700,"props":2874,"children":2875},{"style":804},[2876],{"type":28,"value":1967},{"type":22,"tag":700,"props":2878,"children":2879},{"class":702,"line":766},[2880,2884],{"type":22,"tag":700,"props":2881,"children":2882},{"style":713},[2883],{"type":28,"value":1975},{"type":22,"tag":700,"props":2885,"children":2886},{"style":719},[2887],{"type":28,"value":1980},{"type":22,"tag":700,"props":2889,"children":2890},{"class":702,"line":780},[2891,2895,2900,2904],{"type":22,"tag":700,"props":2892,"children":2893},{"style":713},[2894],{"type":28,"value":1975},{"type":22,"tag":700,"props":2896,"children":2897},{"style":719},[2898],{"type":28,"value":2899},"::WindowingFunction",{"type":22,"tag":700,"props":2901,"children":2902},{"style":707},[2903],{"type":28,"value":2563},{"type":22,"tag":700,"props":2905,"children":2906},{"style":719},[2907],{"type":28,"value":2908}," fWindow;\n",{"type":22,"tag":700,"props":2910,"children":2911},{"class":702,"line":810},[2912],{"type":22,"tag":700,"props":2913,"children":2914},{"style":804},[2915],{"type":28,"value":1967},{"type":22,"tag":700,"props":2917,"children":2918},{"class":702,"line":819},[2919],{"type":22,"tag":700,"props":2920,"children":2921},{"style":719},[2922],{"type":28,"value":1995},{"type":22,"tag":700,"props":2924,"children":2925},{"class":702,"line":1121},[2926],{"type":22,"tag":700,"props":2927,"children":2928},{"emptyLinePlaceholder":1011},[2929],{"type":28,"value":1014},{"type":22,"tag":700,"props":2931,"children":2932},{"class":702,"line":1129},[2933,2937,2941,2945,2949,2953,2957,2961],{"type":22,"tag":700,"props":2934,"children":2935},{"style":713},[2936],{"type":28,"value":2010},{"type":22,"tag":700,"props":2938,"children":2939},{"style":719},[2940],{"type":28,"value":910},{"type":22,"tag":700,"props":2942,"children":2943},{"style":713},[2944],{"type":28,"value":2010},{"type":22,"tag":700,"props":2946,"children":2947},{"style":719},[2948],{"type":28,"value":791},{"type":22,"tag":700,"props":2950,"children":2951},{"style":707},[2952],{"type":28,"value":2027},{"type":22,"tag":700,"props":2954,"children":2955},{"style":719},[2956],{"type":28,"value":2032},{"type":22,"tag":700,"props":2958,"children":2959},{"style":804},[2960],{"type":28,"value":2037},{"type":22,"tag":700,"props":2962,"children":2963},{"style":719},[2964],{"type":28,"value":754},{"type":22,"tag":700,"props":2966,"children":2967},{"class":702,"line":1164},[2968,2972,2976,2980,2984,2988,2992],{"type":22,"tag":700,"props":2969,"children":2970},{"style":719},[2971],{"type":28,"value":2049},{"type":22,"tag":700,"props":2973,"children":2974},{"style":713},[2975],{"type":28,"value":2054},{"type":22,"tag":700,"props":2977,"children":2978},{"style":719},[2979],{"type":28,"value":791},{"type":22,"tag":700,"props":2981,"children":2982},{"style":713},[2983],{"type":28,"value":2063},{"type":22,"tag":700,"props":2985,"children":2986},{"style":719},[2987],{"type":28,"value":910},{"type":22,"tag":700,"props":2989,"children":2990},{"style":713},[2991],{"type":28,"value":2072},{"type":22,"tag":700,"props":2993,"children":2994},{"style":719},[2995],{"type":28,"value":2077},{"type":22,"tag":700,"props":2997,"children":2998},{"class":702,"line":1184},[2999,3003,3008,3013,3017,3021,3025,3029,3033,3037,3041,3046,3050,3054,3059,3064],{"type":22,"tag":700,"props":3000,"children":3001},{"style":719},[3002],{"type":28,"value":2230},{"type":22,"tag":700,"props":3004,"children":3005},{"style":713},[3006],{"type":28,"value":3007},"fWindow",{"type":22,"tag":700,"props":3009,"children":3010},{"style":719},[3011],{"type":28,"value":3012},"(fFft.",{"type":22,"tag":700,"props":3014,"children":3015},{"style":713},[3016],{"type":28,"value":2372},{"type":22,"tag":700,"props":3018,"children":3019},{"style":719},[3020],{"type":28,"value":1302},{"type":22,"tag":700,"props":3022,"children":3023},{"style":707},[3024],{"type":28,"value":1365},{"type":22,"tag":700,"props":3026,"children":3027},{"style":746},[3028],{"type":28,"value":1065},{"type":22,"tag":700,"props":3030,"children":3031},{"style":719},[3032],{"type":28,"value":2230},{"type":22,"tag":700,"props":3034,"children":3035},{"style":713},[3036],{"type":28,"value":1877},{"type":22,"tag":700,"props":3038,"children":3039},{"style":719},[3040],{"type":28,"value":910},{"type":22,"tag":700,"props":3042,"children":3043},{"style":713},[3044],{"type":28,"value":3045},"WindowingFunction",{"type":22,"tag":700,"props":3047,"children":3048},{"style":719},[3049],{"type":28,"value":1258},{"type":22,"tag":700,"props":3051,"children":3052},{"style":707},[3053],{"type":28,"value":2162},{"type":22,"tag":700,"props":3055,"children":3056},{"style":719},[3057],{"type":28,"value":3058},">::hann, ",{"type":22,"tag":700,"props":3060,"children":3061},{"style":746},[3062],{"type":28,"value":3063},"false",{"type":22,"tag":700,"props":3065,"children":3066},{"style":719},[3067],{"type":28,"value":754},{"type":22,"tag":700,"props":3069,"children":3070},{"class":702,"line":1192},[3071],{"type":22,"tag":700,"props":3072,"children":3073},{"style":804},[3074],{"type":28,"value":2085},{"type":22,"tag":700,"props":3076,"children":3077},{"class":702,"line":1230},[3078],{"type":22,"tag":700,"props":3079,"children":3080},{"style":719},[3081],{"type":28,"value":730},{"type":22,"tag":700,"props":3083,"children":3084},{"class":702,"line":1281},[3085],{"type":22,"tag":700,"props":3086,"children":3087},{"style":719},[3088],{"type":28,"value":825},{"type":22,"tag":31,"props":3090,"children":3091},{},[3092],{"type":28,"value":3093},"Now apply the windowing to the chunk of samples before passing it to the FFT.",{"type":22,"tag":274,"props":3095,"children":3097},{"className":692,"code":3096,"language":694,"meta":8,"style":8},"ASpectrogram ASpectroMaker::STFT(const AudioSampleBuffer& signal, size_t hop)\n{\n   // ...Initialize variables as before...\n\n   // While data remains\n   {\n      std::memcpy(fftBuffer.data(), data, fftSize * sizeof(float));\n\n      fWindow.multiplyWithWindowingTable(fftBuffer.data(), fftSize);\n      fFft.performFrequencyOnlyForwardTransform(fftBuffer.data());\n\n      // ...copy the frequency information from fftBuffer to the spectrum\n\n      // Next chunk start\n      data += hop;\n   }\n   //...\n}\n",[3098],{"type":22,"tag":155,"props":3099,"children":3100},{"__ignoreMap":8},[3101,3156,3163,3171,3178,3185,3192,3239,3246,3272,3295,3302,3309,3316,3324,3339,3346,3353],{"type":22,"tag":700,"props":3102,"children":3103},{"class":702,"line":703},[3104,3108,3112,3116,3120,3124,3128,3132,3136,3140,3144,3148,3152],{"type":22,"tag":700,"props":3105,"children":3106},{"style":713},[3107],{"type":28,"value":2187},{"type":22,"tag":700,"props":3109,"children":3110},{"style":713},[3111],{"type":28,"value":2192},{"type":22,"tag":700,"props":3113,"children":3114},{"style":719},[3115],{"type":28,"value":910},{"type":22,"tag":700,"props":3117,"children":3118},{"style":713},[3119],{"type":28,"value":1864},{"type":22,"tag":700,"props":3121,"children":3122},{"style":719},[3123],{"type":28,"value":791},{"type":22,"tag":700,"props":3125,"children":3126},{"style":707},[3127],{"type":28,"value":2209},{"type":22,"tag":700,"props":3129,"children":3130},{"style":713},[3131],{"type":28,"value":2214},{"type":22,"tag":700,"props":3133,"children":3134},{"style":707},[3135],{"type":28,"value":2219},{"type":22,"tag":700,"props":3137,"children":3138},{"style":2222},[3139],{"type":28,"value":2225},{"type":22,"tag":700,"props":3141,"children":3142},{"style":719},[3143],{"type":28,"value":2230},{"type":22,"tag":700,"props":3145,"children":3146},{"style":707},[3147],{"type":28,"value":2235},{"type":22,"tag":700,"props":3149,"children":3150},{"style":2222},[3151],{"type":28,"value":2240},{"type":22,"tag":700,"props":3153,"children":3154},{"style":719},[3155],{"type":28,"value":754},{"type":22,"tag":700,"props":3157,"children":3158},{"class":702,"line":390},[3159],{"type":22,"tag":700,"props":3160,"children":3161},{"style":719},[3162],{"type":28,"value":730},{"type":22,"tag":700,"props":3164,"children":3165},{"class":702,"line":387},[3166],{"type":22,"tag":700,"props":3167,"children":3168},{"style":804},[3169],{"type":28,"value":3170},"   // ...Initialize variables as before...\n",{"type":22,"tag":700,"props":3172,"children":3173},{"class":702,"line":757},[3174],{"type":22,"tag":700,"props":3175,"children":3176},{"emptyLinePlaceholder":1011},[3177],{"type":28,"value":1014},{"type":22,"tag":700,"props":3179,"children":3180},{"class":702,"line":766},[3181],{"type":22,"tag":700,"props":3182,"children":3183},{"style":804},[3184],{"type":28,"value":2605},{"type":22,"tag":700,"props":3186,"children":3187},{"class":702,"line":780},[3188],{"type":22,"tag":700,"props":3189,"children":3190},{"style":719},[3191],{"type":28,"value":763},{"type":22,"tag":700,"props":3193,"children":3194},{"class":702,"line":810},[3195,3199,3203,3207,3211,3215,3219,3223,3227,3231,3235],{"type":22,"tag":700,"props":3196,"children":3197},{"style":713},[3198],{"type":28,"value":1478},{"type":22,"tag":700,"props":3200,"children":3201},{"style":719},[3202],{"type":28,"value":910},{"type":22,"tag":700,"props":3204,"children":3205},{"style":713},[3206],{"type":28,"value":2628},{"type":22,"tag":700,"props":3208,"children":3209},{"style":719},[3210],{"type":28,"value":2633},{"type":22,"tag":700,"props":3212,"children":3213},{"style":713},[3214],{"type":28,"value":2638},{"type":22,"tag":700,"props":3216,"children":3217},{"style":719},[3218],{"type":28,"value":2643},{"type":22,"tag":700,"props":3220,"children":3221},{"style":707},[3222],{"type":28,"value":2578},{"type":22,"tag":700,"props":3224,"children":3225},{"style":707},[3226],{"type":28,"value":2652},{"type":22,"tag":700,"props":3228,"children":3229},{"style":719},[3230],{"type":28,"value":791},{"type":22,"tag":700,"props":3232,"children":3233},{"style":707},[3234],{"type":28,"value":2162},{"type":22,"tag":700,"props":3236,"children":3237},{"style":719},[3238],{"type":28,"value":2665},{"type":22,"tag":700,"props":3240,"children":3241},{"class":702,"line":819},[3242],{"type":22,"tag":700,"props":3243,"children":3244},{"emptyLinePlaceholder":1011},[3245],{"type":28,"value":1014},{"type":22,"tag":700,"props":3247,"children":3248},{"class":702,"line":1121},[3249,3254,3259,3263,3267],{"type":22,"tag":700,"props":3250,"children":3251},{"style":719},[3252],{"type":28,"value":3253},"      fWindow.",{"type":22,"tag":700,"props":3255,"children":3256},{"style":713},[3257],{"type":28,"value":3258},"multiplyWithWindowingTable",{"type":22,"tag":700,"props":3260,"children":3261},{"style":719},[3262],{"type":28,"value":2633},{"type":22,"tag":700,"props":3264,"children":3265},{"style":713},[3266],{"type":28,"value":2638},{"type":22,"tag":700,"props":3268,"children":3269},{"style":719},[3270],{"type":28,"value":3271},"(), fftSize);\n",{"type":22,"tag":700,"props":3273,"children":3274},{"class":702,"line":1129},[3275,3279,3283,3287,3291],{"type":22,"tag":700,"props":3276,"children":3277},{"style":719},[3278],{"type":28,"value":2695},{"type":22,"tag":700,"props":3280,"children":3281},{"style":713},[3282],{"type":28,"value":2700},{"type":22,"tag":700,"props":3284,"children":3285},{"style":719},[3286],{"type":28,"value":2633},{"type":22,"tag":700,"props":3288,"children":3289},{"style":713},[3290],{"type":28,"value":2638},{"type":22,"tag":700,"props":3292,"children":3293},{"style":719},[3294],{"type":28,"value":2713},{"type":22,"tag":700,"props":3296,"children":3297},{"class":702,"line":1164},[3298],{"type":22,"tag":700,"props":3299,"children":3300},{"emptyLinePlaceholder":1011},[3301],{"type":28,"value":1014},{"type":22,"tag":700,"props":3303,"children":3304},{"class":702,"line":1184},[3305],{"type":22,"tag":700,"props":3306,"children":3307},{"style":804},[3308],{"type":28,"value":2730},{"type":22,"tag":700,"props":3310,"children":3311},{"class":702,"line":1192},[3312],{"type":22,"tag":700,"props":3313,"children":3314},{"emptyLinePlaceholder":1011},[3315],{"type":28,"value":1014},{"type":22,"tag":700,"props":3317,"children":3318},{"class":702,"line":1230},[3319],{"type":22,"tag":700,"props":3320,"children":3321},{"style":804},[3322],{"type":28,"value":3323},"      // Next chunk start\n",{"type":22,"tag":700,"props":3325,"children":3326},{"class":702,"line":1281},[3327,3331,3335],{"type":22,"tag":700,"props":3328,"children":3329},{"style":719},[3330],{"type":28,"value":2756},{"type":22,"tag":700,"props":3332,"children":3333},{"style":707},[3334],{"type":28,"value":2761},{"type":22,"tag":700,"props":3336,"children":3337},{"style":719},[3338],{"type":28,"value":2766},{"type":22,"tag":700,"props":3340,"children":3341},{"class":702,"line":1324},[3342],{"type":22,"tag":700,"props":3343,"children":3344},{"style":719},[3345],{"type":28,"value":816},{"type":22,"tag":700,"props":3347,"children":3348},{"class":702,"line":1333},[3349],{"type":22,"tag":700,"props":3350,"children":3351},{"style":804},[3352],{"type":28,"value":1967},{"type":22,"tag":700,"props":3354,"children":3355},{"class":702,"line":1346},[3356],{"type":22,"tag":700,"props":3357,"children":3358},{"style":719},[3359],{"type":28,"value":825},{"type":22,"tag":31,"props":3361,"children":3362},{},[3363,3365,3371,3373,3378],{"type":28,"value":3364},"I'll skip how the data is read into the ",{"type":22,"tag":155,"props":3366,"children":3368},{"className":3367},[],[3369],{"type":28,"value":3370},"Spectrum",{"type":28,"value":3372}," where ",{"type":22,"tag":155,"props":3374,"children":3376},{"className":3375},[],[3377],{"type":28,"value":2187},{"type":28,"value":3379}," keeps per-channel information, but it is straightforward. Just remember to discard the bins corresponding to negative frequencies. And we flip the data vertically so that higher frequencies are at the top, lower frequencies are at the bottom.",{"type":22,"tag":31,"props":3381,"children":3382},{},[3383],{"type":28,"value":3384},"There's a more interesting thing we need to do after that.",{"type":22,"tag":59,"props":3386,"children":3388},{"id":3387},"we-dont-hear-hertz",[3389],{"type":28,"value":3390},"We Don't Hear Hertz",{"type":22,"tag":31,"props":3392,"children":3393},{},[3394,3396,3403],{"type":28,"value":3395},"The FFT bins are ranges of Hertz (Hz). Hertz are a linear representation of frequency, but our ",{"type":22,"tag":110,"props":3397,"children":3400},{"href":3398,"rel":3399},"https://en.wikipedia.org/wiki/Psychoacoustics#/media/File:Perceived_Human_Hearing.svg",[114],[3401],{"type":28,"value":3402},"hearing's perception",{"type":28,"value":3404}," is logarithmic. That means we don't perceive much difference between 10,000 Hz and 10,500 Hz, but by 20,000 Hz we will. It also means we hear a lot of change between 20 Hz and 500 Hz. A lot of the resolution in the linear spacing of the Hz bins is wasted given our perception.",{"type":22,"tag":31,"props":3406,"children":3407},{},[3408,3410,3417],{"type":28,"value":3409},"In order that our classification system finds features in the spectrogram that are more likely to correspond to the way we hear the sound, we should convert from Hertz to another measurement that matches our hearing. For the method Dr. Hawley brought us that is... well it's commonly called ",{"type":22,"tag":110,"props":3411,"children":3414},{"href":3412,"rel":3413},"https://en.wikipedia.org/wiki/Mel-frequency_cepstrum",[114],[3415],{"type":28,"value":3416},"Mels",{"type":28,"value":3418},". I won't presume to understand or explain everything that is going on in the conversion to Mels, but it boils down to a matrix multiplication with a specially constructed conversion matrix that stretches and squeezes information from our evenly spaced linear Hertz bins into bins that better represent our perception of the sound's frequencies.",{"type":22,"tag":31,"props":3420,"children":3421},{},[3422,3424,3430,3432,3438,3440,3446,3448,3454,3456,3462,3464,3470],{"type":28,"value":3423},"The conversion matrix ends up being ",{"type":22,"tag":155,"props":3425,"children":3427},{"className":3426},[],[3428],{"type":28,"value":3429},"fftSize x mels",{"type":28,"value":3431},". Multiplying the frequency data which is ",{"type":22,"tag":155,"props":3433,"children":3435},{"className":3434},[],[3436],{"type":28,"value":3437},"n x fftsize",{"type":28,"value":3439}," with it will produce the ",{"type":22,"tag":155,"props":3441,"children":3443},{"className":3442},[],[3444],{"type":28,"value":3445},"n x mels",{"type":28,"value":3447}," desired output where ",{"type":22,"tag":155,"props":3449,"children":3451},{"className":3450},[],[3452],{"type":28,"value":3453},"n",{"type":28,"value":3455}," is the width of our spectrogram. And we reduce the total size of the spectrogram data by as much. In our application we go from the Hz representation ",{"type":22,"tag":155,"props":3457,"children":3459},{"className":3458},[],[3460],{"type":28,"value":3461},"1025 x 259",{"type":28,"value":3463}," to ",{"type":22,"tag":155,"props":3465,"children":3467},{"className":3466},[],[3468],{"type":28,"value":3469},"96 x 259",{"type":28,"value":3471}," in the Mels representation.",{"type":22,"tag":59,"props":3473,"children":3475},{"id":3474},"the-final-transforms",[3476],{"type":28,"value":3477},"The Final Transforms",{"type":22,"tag":31,"props":3479,"children":3480},{},[3481,3483,3488,3490,3496],{"type":28,"value":3482},"Once we have converted to Mels, we make a final transformation from amplitudes to deciBels (db) while normalizing and clamping to a maximum dB. To use the ",{"type":22,"tag":37,"props":3484,"children":3485},{},[3486],{"type":28,"value":3487},"power to dB",{"type":28,"value":3489}," formula we need powers rather than amplitudes. This just means squaring the signal, i.e. multiply it by itself. Notice that in the ",{"type":22,"tag":155,"props":3491,"children":3493},{"className":3492},[],[3494],{"type":28,"value":3495},"power-to-db",{"type":28,"value":3497}," code it again squares the values. It needs both.",{"type":22,"tag":274,"props":3499,"children":3501},{"className":692,"code":3500,"language":694,"meta":8,"style":8},"   // TransformChannels will modify the data in each channel (aka Spectrum) in the spectrogram in place.\n   // Power to dB while limiting to a minimum magnitude.\n   TransformChannels(magnitudes, [](float mag)\n    {\n       constexpr double kMin = 1e-5 * 1e-5;\n       constexpr double kReference = 1.0f * 1.0f;\n       return 10.0 * std::log10(std::max(kMin, (double)mag * mag))\n            - 10.0 * std::log10(std::max(kMin, kReference));\n    });\n\n   // Limit to maximum dB\n   auto maxDb = magnitudes.MinMax().second - kMaxDb;\n   TransformChannels(magnitudes, [maxDb](float dB)\n    {\n       return std::max(dB, maxDb);\n    });\n",[3502],{"type":22,"tag":155,"props":3503,"children":3504},{"__ignoreMap":8},[3505,3513,3521,3547,3555,3612,3658,3729,3778,3786,3793,3801,3841,3876,3883,3907],{"type":22,"tag":700,"props":3506,"children":3507},{"class":702,"line":703},[3508],{"type":22,"tag":700,"props":3509,"children":3510},{"style":804},[3511],{"type":28,"value":3512},"   // TransformChannels will modify the data in each channel (aka Spectrum) in the spectrogram in place.\n",{"type":22,"tag":700,"props":3514,"children":3515},{"class":702,"line":390},[3516],{"type":22,"tag":700,"props":3517,"children":3518},{"style":804},[3519],{"type":28,"value":3520},"   // Power to dB while limiting to a minimum magnitude.\n",{"type":22,"tag":700,"props":3522,"children":3523},{"class":702,"line":387},[3524,3529,3534,3538,3543],{"type":22,"tag":700,"props":3525,"children":3526},{"style":713},[3527],{"type":28,"value":3528},"   TransformChannels",{"type":22,"tag":700,"props":3530,"children":3531},{"style":719},[3532],{"type":28,"value":3533},"(magnitudes, [](",{"type":22,"tag":700,"props":3535,"children":3536},{"style":707},[3537],{"type":28,"value":2162},{"type":22,"tag":700,"props":3539,"children":3540},{"style":2222},[3541],{"type":28,"value":3542}," mag",{"type":22,"tag":700,"props":3544,"children":3545},{"style":719},[3546],{"type":28,"value":754},{"type":22,"tag":700,"props":3548,"children":3549},{"class":702,"line":757},[3550],{"type":22,"tag":700,"props":3551,"children":3552},{"style":719},[3553],{"type":28,"value":3554},"    {\n",{"type":22,"tag":700,"props":3556,"children":3557},{"class":702,"line":766},[3558,3563,3568,3573,3577,3581,3586,3591,3596,3600,3604,3608],{"type":22,"tag":700,"props":3559,"children":3560},{"style":707},[3561],{"type":28,"value":3562},"       constexpr",{"type":22,"tag":700,"props":3564,"children":3565},{"style":707},[3566],{"type":28,"value":3567}," double",{"type":22,"tag":700,"props":3569,"children":3570},{"style":719},[3571],{"type":28,"value":3572}," kMin ",{"type":22,"tag":700,"props":3574,"children":3575},{"style":707},[3576],{"type":28,"value":1060},{"type":22,"tag":700,"props":3578,"children":3579},{"style":746},[3580],{"type":28,"value":1065},{"type":22,"tag":700,"props":3582,"children":3583},{"style":707},[3584],{"type":28,"value":3585},"e-",{"type":22,"tag":700,"props":3587,"children":3588},{"style":746},[3589],{"type":28,"value":3590},"5",{"type":22,"tag":700,"props":3592,"children":3593},{"style":707},[3594],{"type":28,"value":3595}," *",{"type":22,"tag":700,"props":3597,"children":3598},{"style":746},[3599],{"type":28,"value":1065},{"type":22,"tag":700,"props":3601,"children":3602},{"style":707},[3603],{"type":28,"value":3585},{"type":22,"tag":700,"props":3605,"children":3606},{"style":746},[3607],{"type":28,"value":3590},{"type":22,"tag":700,"props":3609,"children":3610},{"style":719},[3611],{"type":28,"value":1075},{"type":22,"tag":700,"props":3613,"children":3614},{"class":702,"line":780},[3615,3619,3623,3628,3632,3637,3642,3646,3650,3654],{"type":22,"tag":700,"props":3616,"children":3617},{"style":707},[3618],{"type":28,"value":3562},{"type":22,"tag":700,"props":3620,"children":3621},{"style":707},[3622],{"type":28,"value":3567},{"type":22,"tag":700,"props":3624,"children":3625},{"style":719},[3626],{"type":28,"value":3627}," kReference ",{"type":22,"tag":700,"props":3629,"children":3630},{"style":707},[3631],{"type":28,"value":1060},{"type":22,"tag":700,"props":3633,"children":3634},{"style":746},[3635],{"type":28,"value":3636}," 1.0",{"type":22,"tag":700,"props":3638,"children":3639},{"style":707},[3640],{"type":28,"value":3641},"f",{"type":22,"tag":700,"props":3643,"children":3644},{"style":707},[3645],{"type":28,"value":3595},{"type":22,"tag":700,"props":3647,"children":3648},{"style":746},[3649],{"type":28,"value":3636},{"type":22,"tag":700,"props":3651,"children":3652},{"style":707},[3653],{"type":28,"value":3641},{"type":22,"tag":700,"props":3655,"children":3656},{"style":719},[3657],{"type":28,"value":1075},{"type":22,"tag":700,"props":3659,"children":3660},{"class":702,"line":810},[3661,3666,3671,3675,3679,3683,3688,3692,3696,3700,3705,3710,3715,3720,3724],{"type":22,"tag":700,"props":3662,"children":3663},{"style":707},[3664],{"type":28,"value":3665},"       return",{"type":22,"tag":700,"props":3667,"children":3668},{"style":746},[3669],{"type":28,"value":3670}," 10.0",{"type":22,"tag":700,"props":3672,"children":3673},{"style":707},[3674],{"type":28,"value":3595},{"type":22,"tag":700,"props":3676,"children":3677},{"style":713},[3678],{"type":28,"value":905},{"type":22,"tag":700,"props":3680,"children":3681},{"style":719},[3682],{"type":28,"value":910},{"type":22,"tag":700,"props":3684,"children":3685},{"style":713},[3686],{"type":28,"value":3687},"log10",{"type":22,"tag":700,"props":3689,"children":3690},{"style":719},[3691],{"type":28,"value":791},{"type":22,"tag":700,"props":3693,"children":3694},{"style":713},[3695],{"type":28,"value":2063},{"type":22,"tag":700,"props":3697,"children":3698},{"style":719},[3699],{"type":28,"value":910},{"type":22,"tag":700,"props":3701,"children":3702},{"style":713},[3703],{"type":28,"value":3704},"max",{"type":22,"tag":700,"props":3706,"children":3707},{"style":719},[3708],{"type":28,"value":3709},"(kMin, (",{"type":22,"tag":700,"props":3711,"children":3712},{"style":707},[3713],{"type":28,"value":3714},"double",{"type":22,"tag":700,"props":3716,"children":3717},{"style":719},[3718],{"type":28,"value":3719},")mag ",{"type":22,"tag":700,"props":3721,"children":3722},{"style":707},[3723],{"type":28,"value":2578},{"type":22,"tag":700,"props":3725,"children":3726},{"style":719},[3727],{"type":28,"value":3728}," mag))\n",{"type":22,"tag":700,"props":3730,"children":3731},{"class":702,"line":819},[3732,3737,3741,3745,3749,3753,3757,3761,3765,3769,3773],{"type":22,"tag":700,"props":3733,"children":3734},{"style":707},[3735],{"type":28,"value":3736},"            -",{"type":22,"tag":700,"props":3738,"children":3739},{"style":746},[3740],{"type":28,"value":3670},{"type":22,"tag":700,"props":3742,"children":3743},{"style":707},[3744],{"type":28,"value":3595},{"type":22,"tag":700,"props":3746,"children":3747},{"style":713},[3748],{"type":28,"value":905},{"type":22,"tag":700,"props":3750,"children":3751},{"style":719},[3752],{"type":28,"value":910},{"type":22,"tag":700,"props":3754,"children":3755},{"style":713},[3756],{"type":28,"value":3687},{"type":22,"tag":700,"props":3758,"children":3759},{"style":719},[3760],{"type":28,"value":791},{"type":22,"tag":700,"props":3762,"children":3763},{"style":713},[3764],{"type":28,"value":2063},{"type":22,"tag":700,"props":3766,"children":3767},{"style":719},[3768],{"type":28,"value":910},{"type":22,"tag":700,"props":3770,"children":3771},{"style":713},[3772],{"type":28,"value":3704},{"type":22,"tag":700,"props":3774,"children":3775},{"style":719},[3776],{"type":28,"value":3777},"(kMin, kReference));\n",{"type":22,"tag":700,"props":3779,"children":3780},{"class":702,"line":1121},[3781],{"type":22,"tag":700,"props":3782,"children":3783},{"style":719},[3784],{"type":28,"value":3785},"    });\n",{"type":22,"tag":700,"props":3787,"children":3788},{"class":702,"line":1129},[3789],{"type":22,"tag":700,"props":3790,"children":3791},{"emptyLinePlaceholder":1011},[3792],{"type":28,"value":1014},{"type":22,"tag":700,"props":3794,"children":3795},{"class":702,"line":1164},[3796],{"type":22,"tag":700,"props":3797,"children":3798},{"style":804},[3799],{"type":28,"value":3800},"   // Limit to maximum dB\n",{"type":22,"tag":700,"props":3802,"children":3803},{"class":702,"line":1184},[3804,3808,3813,3817,3822,3827,3832,3836],{"type":22,"tag":700,"props":3805,"children":3806},{"style":707},[3807],{"type":28,"value":1135},{"type":22,"tag":700,"props":3809,"children":3810},{"style":719},[3811],{"type":28,"value":3812}," maxDb ",{"type":22,"tag":700,"props":3814,"children":3815},{"style":707},[3816],{"type":28,"value":1060},{"type":22,"tag":700,"props":3818,"children":3819},{"style":719},[3820],{"type":28,"value":3821}," magnitudes.",{"type":22,"tag":700,"props":3823,"children":3824},{"style":713},[3825],{"type":28,"value":3826},"MinMax",{"type":22,"tag":700,"props":3828,"children":3829},{"style":719},[3830],{"type":28,"value":3831},"().second ",{"type":22,"tag":700,"props":3833,"children":3834},{"style":707},[3835],{"type":28,"value":1273},{"type":22,"tag":700,"props":3837,"children":3838},{"style":719},[3839],{"type":28,"value":3840}," kMaxDb;\n",{"type":22,"tag":700,"props":3842,"children":3843},{"class":702,"line":1192},[3844,3848,3853,3858,3863,3867,3872],{"type":22,"tag":700,"props":3845,"children":3846},{"style":713},[3847],{"type":28,"value":3528},{"type":22,"tag":700,"props":3849,"children":3850},{"style":719},[3851],{"type":28,"value":3852},"(magnitudes, [",{"type":22,"tag":700,"props":3854,"children":3855},{"style":2222},[3856],{"type":28,"value":3857},"maxDb",{"type":22,"tag":700,"props":3859,"children":3860},{"style":719},[3861],{"type":28,"value":3862},"](",{"type":22,"tag":700,"props":3864,"children":3865},{"style":707},[3866],{"type":28,"value":2162},{"type":22,"tag":700,"props":3868,"children":3869},{"style":2222},[3870],{"type":28,"value":3871}," dB",{"type":22,"tag":700,"props":3873,"children":3874},{"style":719},[3875],{"type":28,"value":754},{"type":22,"tag":700,"props":3877,"children":3878},{"class":702,"line":1230},[3879],{"type":22,"tag":700,"props":3880,"children":3881},{"style":719},[3882],{"type":28,"value":3554},{"type":22,"tag":700,"props":3884,"children":3885},{"class":702,"line":1281},[3886,3890,3894,3898,3902],{"type":22,"tag":700,"props":3887,"children":3888},{"style":707},[3889],{"type":28,"value":3665},{"type":22,"tag":700,"props":3891,"children":3892},{"style":713},[3893],{"type":28,"value":905},{"type":22,"tag":700,"props":3895,"children":3896},{"style":719},[3897],{"type":28,"value":910},{"type":22,"tag":700,"props":3899,"children":3900},{"style":713},[3901],{"type":28,"value":3704},{"type":22,"tag":700,"props":3903,"children":3904},{"style":719},[3905],{"type":28,"value":3906},"(dB, maxDb);\n",{"type":22,"tag":700,"props":3908,"children":3909},{"class":702,"line":1324},[3910],{"type":22,"tag":700,"props":3911,"children":3912},{"style":719},[3913],{"type":28,"value":3785},{"type":22,"tag":59,"props":3915,"children":3917},{"id":3916},"view-from-the-top",[3918],{"type":28,"value":3919},"View From The Top",{"type":22,"tag":31,"props":3921,"children":3922},{},[3923,3925,3930,3932,3938],{"type":28,"value":3924},"Here is the top-level ",{"type":22,"tag":155,"props":3926,"children":3928},{"className":3927},[],[3929],{"type":28,"value":2010},{"type":28,"value":3931}," code snippet that goes from audio file to the final spectrogram. When it finishes ",{"type":22,"tag":155,"props":3933,"children":3935},{"className":3934},[],[3936],{"type":28,"value":3937},"spectrogram",{"type":28,"value":3939}," contains the final Mel frequency information.",{"type":22,"tag":274,"props":3941,"children":3943},{"className":692,"code":3942,"language":694,"meta":8,"style":8},"   // Read a little more than 3 seconds to get the amount of data needed to produce 259 columns in the spectrogram\n   auto signal = this->ReadAudioSignal(file, 3.1);\n   auto hzFreqs = this->STFT(ToMono(signal).buffer, kHopLength);\n   TransformChannels(hzFreqs, [](auto x) { return x * x; });\n\n   spectrogram.AddChannel(ASpectroMaker::HzToMels(hzFreqs, 0, signal.sampleRate, fNumMelBins));\n   spectrogram.Reshape(fNumMelBins, hzFreqs.Shape().second);\n   ASpectroMaker::AmplitudesToDbs(spectrogram);\n",[3944],{"type":22,"tag":155,"props":3945,"children":3946},{"__ignoreMap":8},[3947,3955,4000,4042,4088,4095,4139,4166],{"type":22,"tag":700,"props":3948,"children":3949},{"class":702,"line":703},[3950],{"type":22,"tag":700,"props":3951,"children":3952},{"style":804},[3953],{"type":28,"value":3954},"   // Read a little more than 3 seconds to get the amount of data needed to produce 259 columns in the spectrogram\n",{"type":22,"tag":700,"props":3956,"children":3957},{"class":702,"line":390},[3958,3962,3967,3971,3976,3981,3986,3991,3996],{"type":22,"tag":700,"props":3959,"children":3960},{"style":707},[3961],{"type":28,"value":1135},{"type":22,"tag":700,"props":3963,"children":3964},{"style":719},[3965],{"type":28,"value":3966}," signal ",{"type":22,"tag":700,"props":3968,"children":3969},{"style":707},[3970],{"type":28,"value":1060},{"type":22,"tag":700,"props":3972,"children":3973},{"style":746},[3974],{"type":28,"value":3975}," this",{"type":22,"tag":700,"props":3977,"children":3978},{"style":719},[3979],{"type":28,"value":3980},"->",{"type":22,"tag":700,"props":3982,"children":3983},{"style":713},[3984],{"type":28,"value":3985},"ReadAudioSignal",{"type":22,"tag":700,"props":3987,"children":3988},{"style":719},[3989],{"type":28,"value":3990},"(file, ",{"type":22,"tag":700,"props":3992,"children":3993},{"style":746},[3994],{"type":28,"value":3995},"3.1",{"type":22,"tag":700,"props":3997,"children":3998},{"style":719},[3999],{"type":28,"value":2296},{"type":22,"tag":700,"props":4001,"children":4002},{"class":702,"line":387},[4003,4007,4012,4016,4020,4024,4028,4032,4037],{"type":22,"tag":700,"props":4004,"children":4005},{"style":707},[4006],{"type":28,"value":1135},{"type":22,"tag":700,"props":4008,"children":4009},{"style":719},[4010],{"type":28,"value":4011}," hzFreqs ",{"type":22,"tag":700,"props":4013,"children":4014},{"style":707},[4015],{"type":28,"value":1060},{"type":22,"tag":700,"props":4017,"children":4018},{"style":746},[4019],{"type":28,"value":3975},{"type":22,"tag":700,"props":4021,"children":4022},{"style":719},[4023],{"type":28,"value":3980},{"type":22,"tag":700,"props":4025,"children":4026},{"style":713},[4027],{"type":28,"value":1864},{"type":22,"tag":700,"props":4029,"children":4030},{"style":719},[4031],{"type":28,"value":791},{"type":22,"tag":700,"props":4033,"children":4034},{"style":713},[4035],{"type":28,"value":4036},"ToMono",{"type":22,"tag":700,"props":4038,"children":4039},{"style":719},[4040],{"type":28,"value":4041},"(signal).buffer, kHopLength);\n",{"type":22,"tag":700,"props":4043,"children":4044},{"class":702,"line":757},[4045,4049,4054,4059,4064,4069,4074,4079,4083],{"type":22,"tag":700,"props":4046,"children":4047},{"style":713},[4048],{"type":28,"value":3528},{"type":22,"tag":700,"props":4050,"children":4051},{"style":719},[4052],{"type":28,"value":4053},"(hzFreqs, [](",{"type":22,"tag":700,"props":4055,"children":4056},{"style":707},[4057],{"type":28,"value":4058},"auto",{"type":22,"tag":700,"props":4060,"children":4061},{"style":2222},[4062],{"type":28,"value":4063}," x",{"type":22,"tag":700,"props":4065,"children":4066},{"style":719},[4067],{"type":28,"value":4068},") { ",{"type":22,"tag":700,"props":4070,"children":4071},{"style":707},[4072],{"type":28,"value":4073},"return",{"type":22,"tag":700,"props":4075,"children":4076},{"style":719},[4077],{"type":28,"value":4078}," x ",{"type":22,"tag":700,"props":4080,"children":4081},{"style":707},[4082],{"type":28,"value":2578},{"type":22,"tag":700,"props":4084,"children":4085},{"style":719},[4086],{"type":28,"value":4087}," x; });\n",{"type":22,"tag":700,"props":4089,"children":4090},{"class":702,"line":766},[4091],{"type":22,"tag":700,"props":4092,"children":4093},{"emptyLinePlaceholder":1011},[4094],{"type":28,"value":1014},{"type":22,"tag":700,"props":4096,"children":4097},{"class":702,"line":780},[4098,4103,4108,4112,4116,4120,4125,4130,4134],{"type":22,"tag":700,"props":4099,"children":4100},{"style":719},[4101],{"type":28,"value":4102},"   spectrogram.",{"type":22,"tag":700,"props":4104,"children":4105},{"style":713},[4106],{"type":28,"value":4107},"AddChannel",{"type":22,"tag":700,"props":4109,"children":4110},{"style":719},[4111],{"type":28,"value":791},{"type":22,"tag":700,"props":4113,"children":4114},{"style":713},[4115],{"type":28,"value":2010},{"type":22,"tag":700,"props":4117,"children":4118},{"style":719},[4119],{"type":28,"value":910},{"type":22,"tag":700,"props":4121,"children":4122},{"style":713},[4123],{"type":28,"value":4124},"HzToMels",{"type":22,"tag":700,"props":4126,"children":4127},{"style":719},[4128],{"type":28,"value":4129},"(hzFreqs, ",{"type":22,"tag":700,"props":4131,"children":4132},{"style":746},[4133],{"type":28,"value":2291},{"type":22,"tag":700,"props":4135,"children":4136},{"style":719},[4137],{"type":28,"value":4138},", signal.sampleRate, fNumMelBins));\n",{"type":22,"tag":700,"props":4140,"children":4141},{"class":702,"line":810},[4142,4146,4151,4156,4161],{"type":22,"tag":700,"props":4143,"children":4144},{"style":719},[4145],{"type":28,"value":4102},{"type":22,"tag":700,"props":4147,"children":4148},{"style":713},[4149],{"type":28,"value":4150},"Reshape",{"type":22,"tag":700,"props":4152,"children":4153},{"style":719},[4154],{"type":28,"value":4155},"(fNumMelBins, hzFreqs.",{"type":22,"tag":700,"props":4157,"children":4158},{"style":713},[4159],{"type":28,"value":4160},"Shape",{"type":22,"tag":700,"props":4162,"children":4163},{"style":719},[4164],{"type":28,"value":4165},"().second);\n",{"type":22,"tag":700,"props":4167,"children":4168},{"class":702,"line":819},[4169,4174,4178,4183],{"type":22,"tag":700,"props":4170,"children":4171},{"style":713},[4172],{"type":28,"value":4173},"   ASpectroMaker",{"type":22,"tag":700,"props":4175,"children":4176},{"style":719},[4177],{"type":28,"value":910},{"type":22,"tag":700,"props":4179,"children":4180},{"style":713},[4181],{"type":28,"value":4182},"AmplitudesToDbs",{"type":22,"tag":700,"props":4184,"children":4185},{"style":719},[4186],{"type":28,"value":4187},"(spectrogram);\n",{"type":22,"tag":59,"props":4189,"children":4191},{"id":4190},"concluding",[4192],{"type":28,"value":4193},"Concluding",{"type":22,"tag":31,"props":4195,"children":4196},{},[4197],{"type":28,"value":4198},"You now have a two-dimensional spectrogram represented as a set of floating point values in a vector. This is suitable to export, say as a NumPy array, or conversion to an actual image representation in color. We do both.",{"type":22,"tag":31,"props":4200,"children":4201},{},[4202],{"type":28,"value":4203},"Hopefully this article has let you get a little better understanding of the how and the why, and you can avoid some of the confusion I experienced during development.",{"type":22,"tag":31,"props":4205,"children":4206},{},[4207,4209,4216,4217,4224,4226,4233],{"type":28,"value":4208},"Some notes.\n* We discard the phase information the FFT can produce; this means we cannot reliably convert back to audio from the spectrogram\n* Dr. Hawley's Python implementation used ",{"type":22,"tag":110,"props":4210,"children":4213},{"href":4211,"rel":4212},"https://librosa.github.io",[114],[4214],{"type":28,"value":4215},"librosa",{"type":28,"value":531},{"type":22,"tag":110,"props":4218,"children":4221},{"href":4219,"rel":4220},"https://numpy.org",[114],[4222],{"type":28,"value":4223},"NumPy",{"type":28,"value":4225}," for much of the audio processing; we based our implementation on that code, and used his parameters\n* The JUCE ",{"type":22,"tag":110,"props":4227,"children":4230},{"href":4228,"rel":4229},"https://docs.juce.com/master/tutorial_simple_fft.html",[114],[4231],{"type":28,"value":4232},"spectrogram demo",{"type":28,"value":4234}," was also very helpful",{"type":22,"tag":1635,"props":4236,"children":4237},{},[4238],{"type":28,"value":1639},{"title":8,"searchDepth":387,"depth":387,"links":4240},[4241,4242,4243,4244,4245,4246,4247,4248],{"id":1824,"depth":390,"text":1827},{"id":1851,"depth":390,"text":1854},{"id":2102,"depth":390,"text":2105},{"id":2793,"depth":390,"text":2796},{"id":3387,"depth":390,"text":3390},{"id":3474,"depth":390,"text":3477},{"id":3916,"depth":390,"text":3919},{"id":4190,"depth":390,"text":4193},"content:jbagley:2019-4:MakingSpectrogramsInJUCE.md","jbagley/2019-4/MakingSpectrogramsInJUCE.md","jbagley/2019-4/MakingSpectrogramsInJUCE",{"user":406,"name":407},1780330263273]