============================================================================================== SYSTEM DESIGN LEARNING VIDEOS AND PDF DOCUMENTS - NOTES ============================================================================================== |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ============================================================================================== [ + ] DEVOPS ============================================================================================== |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ============================================================================================== [ > ]:::: DevOps Master Class - Part 1 - Foundation ============================================================================================== + DevOps: people.process.products: enable continuous delivery of value to customers + (changes) Developers >(value(Business)value)< Operations (lights_on) + requirements > project > delivery # unite dev.ops [ plan + track ] >> [ develop ] >> build >> test.release.deploy.operate >> monitor/learn <<< (( loop time )) = speed to react # monitor user behaviour (+) where is most value = good info >> plan & track (+) [[[[monolithAPP]]]]: (microservices)-loose_coupling-containers-serverless-(REST)==cloud (+) cloud_$_consumption-based : shift in architecture shift in the hosting (+) waterfall: customer > requirements > design > develop > test > deliver >> customer << (+) Agile : small-incremental-value : design.dev.test.deliver : customer-interactions (+) individuals (+) interaction (+) customer-collaboration (+) respond-to-change [[[SCRUM]]:customer.requirement.product-backlog:[[SPRINT]]build.develop.test.deploy[sprint] [[[SCRUM]]:daily-standup:fixturation:pull-work|doing|done| [[[KANBAN]]]:customer.requirement.product-backlog:continuous-stream-of-work [[[KANBAN]]]:WIPlimit:build(3i).test(2i).deploy(3i):pull-when-have-space (no-50things-to-test) [[[KANBAN]]]start-how-you-work/incremCHG/respctROLE/leadershipALL/visualBOARD/limitWIP// //alwaysPULL/goodDEFNTN/improveCOLLAB: bottlenecks:meetings-feedback:pull:more-active AzureDevOps: Epics/Features/PBI : ScrumBan : SCRUM-to-Kanban : show the value [KANBAN-pizza] = product-backlog[][][][] =order= table-pizzas(WIPlimit)_pull[] =cook= {{OVEN}}(WIPlimi)_pull{%} =check/box= _pull =deliver= [SCRUM-pizza] = 30minSprint = 60pizzapoints = [][][]30mins_pull =whatwhentwell= sprint/retro (+) GIT & CI/CD : concurrentDEV : codeBRANCH : GITrepo : [repo-full-copy]synch : distributed (+) GITHUB : no-big-long-gaps : frequentCHGs : CI : commits : continuous_integration (+) ()__pipeline__): security.build.test.image.uploadRGSTRY : frequently-commit (+) CD/CD : ()_continuous_delivery_) : build-infra.artifact.uatINFRA.devINFRA.testINFRA (+) CD/CD : ()_continuous_deployment_) : prodINFRA.prodINFRAdev.prodINFRAtest <*canary CI/CD (+) VAULT : secrets/keys/certs : RBAC : AKV : pipeline-integration (+) MONITOR : user-behaviour : telemetry : KPI : sentiment : #admins : SLA (+) IaC [yaml][json] : Declarative : Idempotent : k8s : OS : gitops-flux-helm ============================================================================================== [ > ]:::: DevOps Master Class - Part 2 - Master Git! ============================================================================================== + Version Control + track with metadata + source of truth + enable collaboration + roll-back + Consistency + History + immutability + additional security + CI/CD + centralized + (( REPO ))--: centralized ||| (( REPO )):distributed(conflicts) + (( GIT )) (entire-copy-of-repo/branches) git-scm.com \git\cmd git.path gitLens VScode # git --version # git init # git status # git config --global --list # --show-origin # # git config --global init.defaultBranch main # blob1 #blob2---file2.txt{#TREE}integrity-feat # COMMIT: certain-point-in-time snapshot of the entire REPO # commit<->|tree|author/parent # sha-1 hash: everything is an object on the file system # BRANCH: pointer to a certain commit # .git__objects # git init # testfile.txt # git add. # .git__object[sha-1/40-char/38charobj] # sharding: every hash has same 2 chars # .git__objects_/<2char>_/<2char> # git -commit # ZLIBcompressed-files # commit a5%%(head->main) # git cat-file -pa5%% = commit # copy file file2 # git add. status # git commit = new commit + new tree object = pointsParent # git cat-file 6a # testfile&testfile2=same_content&blob(43) # tree:6a/parent:a5/blob:43 # .git__obj__refs_/heads{main}__/HEAD...{main}...#commit | git/refs/heads/main/23%hash # .git/HEAD={refs:refs/heads/main} # git stores 40-char objects # tree # commit:full-copy-repo # (( REPO ))[.git_folder] <#STAGE#> [working_dir]:file.txt # git add. (WD>STG) # git commit (STG>REPO) # git add testfile.txt (WD>STG) # git commit -m # git status # gitgraph = (HEAD->main) 23%testfile2.txt added a5%initial testfile.txt commit # git diff a5%..30% (green/red|lines) # git diff --cached = REPO~STG # git diff = STG~REPO # gitgraph # git rm testfile4 (STG)# git commit (REPO)# git add. (WD)toRemoveFromWD # git reset = STGfolder-match-last-commit # git status = chg-not-staged-for-commit # git reset --hard = also-chg-WD-to-match # git restore --staged tstfile.txt # git restore(WD) # HEAD...main...commit # {#1commit}--{#2commit}--{#3commit}--{#4commit}<-[main]<-[HEAD] # remove a commit from history : git reset HEAD~1 --soft = move "<-[main]<-[HEAD]" to #3 # git reset # --soft(comm-bk) --mixed(STGbk) --hard(WDbk) # git tag v1234 # git tag --list # [REPO_local]-_remote_origin_-->(REPO_git) # git clone # git remote origin add # (#1)(2)(3).main.Head # git push -u # (1)(2)(3).{4inGIT}main.HEAD # git pull = fetch+merge # (#5_loc)--(REPO_git) # .gitignore = *.log # HEAD>m>(#1) # main---(commit)-\____/-- # trunk-based-development (branches-back-to-main) : prevent conflicts # feature-based-devl # git switch branch1/2 # (#1)---(#2)-branch1-HEAD = main still with (#1) = catch main to BR1 # git switch main # git merge branch1 # git merge --no-ff BRnch1 # (#1)<-(#4)-(#3) # git merge = conflicts # resolve and merge # gitgraph shows branches # rebase >> # rebase my branch (#1)--(#4)___(#5)-(#6) # never rebase a Public REPO ! # pull request (REPO)--(#2)-->(#1)::fork::own-copy-of-repo # terminal:clone # # clone -- 2 commits --BRnch --- push # pull request # BRnch into othr base BRnch # merge pull request # merge # protect/settings: branch protection rules : lock ============================================================================================== [ > ]:::: DevOps Master Class - Part 3 - Azure DevOps and GitHub ============================================================================================== >plan->develop->collab->deliv->operate->plan< # GitHub # ADO::::GitHub + ADO = Boards / Repos / Pipelines / Test Plans / Artifacts # GitHub = REPOs/projects + ADO = GitHub_connections # AB_commit(AzureBoard_link_to_GitHub_commit) + EPIC = BusinessCase/EPIC = OrgLevelStrategy___EPIC__/Features__/UserStories(PBI=unit-value) + ADO = AzureBoards = WorkItems+Boards = Epic/Features/PBIs|WorkItem|UserStory/Tasks + ADO Boards = WorkItems/Features|HighPriorities # Backlogs # Sprint # Queries + GITHUB = [REPO] = issue(Labels) # noPBI-noFeat-noEpic # GitHub_label:noHierarchy + GITHUB # Label & Milestones # ADO_tasks # GITHUB = newIssue = tasks = checkboxes = mrkpLang + GITHUB # Projects = +add_card # add labels # add milestones # create templates # customize + GITHUB = all files in my repo = template = commit # NewREPO+settings:TemplateRepository + GITHUB = turn_on_discussions # VisualStudio | VSCode | GITHUBcodespaces | codespaces:orgGIT + GITHUB = LiveShare:extension_work_same_piece_of_code # GitHub_copilot:AI/repos + ADO REPOS = files/branching/pull-requests # GITHUB-REPOS:advanced_security:dependencies + GITHUB REPOS = inner source + adv sec + all objects : ()---main---()\branches + GITHUB : branch-protection-rules : secret-scanning : Security/dependabots : SecPolicy + Deliver: CI/CD [REPO]:commit--event--()__CI___):sec:build:test:image:push>registry + [REPO]:pull-request-merge ()__build-off-of-main__):deployPROD # ()_GATES|gates|gate__) + ()_PIPELINES_):stages:gates:triggers:pre-depl-approvals > push the code through:ADOagents + ADOagents / GITHUBrunners : GITHUBactions:anyEvents/notJustCI/CD : WorkflowTemplate + worflow .yml CI on: push: branches:[main] pull_request: branches:[main] #DeplAzureWebApp + Environments: protection-rules: add reviewers | rather than workflows + ADOartifacts vs GITpackageMGMT : ADO:pipelines(CI)/releases(CD) + Monitoring : ENV + APP/user # ADO: dashboards: cycles/work-items/performance + GITHUBinsights: dependency-graph # Grafana # ADO merge with GITHUB + GITHUBroadmap vs ADOfeatureTimeline : ADO__move_repo_to_GITHUB ============================================================================================== [ > ]:::: DevOps Master Class Part 4 CICD ============================================================================================== + Plan ... Develop ... Collab ( GIT ) ... Deliver ( CI/CD ) ... Operate ... Plan + _____/"/""/""feature-branch""\___main # reduce risk of wrong deployment patterns + CI : commit : how do I know I do not introduce a new problem. Does code still work ? + CI/CD : Build/Test : confidence : artifact + VM:[agent/runners]:steps # CI dev__[REPO]|()_pipeline_)npm.nugget.maven + install-dependency ()___build___)test # failure:trigger-something:work-item(bug) + ()__build__test__)create-artifact__)upload_)package_)""registry"" + [devlpr]---[make-release]-.zip--^^--registry : 'security-shift-left' before PRD + DEV+security: check-code / Secrets / Dependencies / Dependabots [Registry]:docker-sec-scan + CD : ContinuousDeployment [registry]--new-artifact-release--()_test_)_staging_)_prod__) + CD : ContinuousDelivery [registry]--new-artifact-release--()_test_)_staging_) + BuildEnvironment: IaC : infrastructure as code : declarative : version control + All environments >>> Production Consistent # Critical to use IaC # Puppet / PS DSC + Kubernetes / YAML : declarative : same package to all env ( conf files ENV ) + ()__testing__) : functionality/security/vulnerabilities/compliance|azure-policies/perfrmnc) + Production Roll-out : deployment strategies {segm_1}{}{}{segm_4}:+simple:down-time/bigbang + deployment slots vs progressive(rings:early-adopters:group1,2,3):+control:complx:taketime + canary (%) : control&simpler vs 'blue/green':flip: resource-usage{create-when-needed()()} + event that makes stuff happen : ADO/Pipelines:build(CI)/release(CD){GUI} Pipelines{YAML} + YAMLpipelines:multistaging:sayhello.yml # name|trigger|pool|[stages]|jobs|[pools]steps/task + [stages]:enablesCD [ENV]:security|resources|apprvl|checks|track|deplmnt-job + ADO:environments=checks : stages|jobs # pools==agents(ms-hosted|self-hosted|long-lived) + GITHUB Actions Event ---trigger-->workflow(jobs):runners[ENV][needs]STEPs .github/workflow + Event: not just CI/CD : commit|pull-req|comment> trigger-workflow + JOBS: runson : runner : github-hosted vm-runner-443-fw-((gh)) + Actions: owner/repo@rep = javascript|container > marketPlace < Azure'actions'ghMarketPlace + runs-on'steps'uses:actions/chk@master ~~> .yml file "deployAKS.yml" #stages|Jobs|needs|ENVs + ENVs : Test Prod Review : wait timers & greetings $secrets.greetings diffVARperENV + approved / queued / deployed # secret*** protect from yourself : greetings ============================================================================================== [ > ]:::: DevOps Master Class - Part 5 - Secrets ============================================================================================== + [RESOURCE]<--auth~secret--[APP]--auth~secret-->((REST)) ()__pipelines__){devops-tooling} + never-put-secrets-in-code: githubREPO:scans for secrets:customizable:add-email-alerts + VM1[function|container[APP]]----(STORAGE) /AAD-IP\{managed-identity}{service-principal} + VM1----/AAD-IP\{managed-identity} # (STORAGE)RBAC:VM1blobReaderRole + VM1----/AAD-IP\{managed-identity} [T]token_to_(STORAGE)_from_AAD-IdentityProvider + StorageAccount[AccessControl]IAM:ReaderRole+StorageBlobDataReader <KVSecretsUser(IAMrbac) + connect-azAccount -identity # new-azstorcont|getAzStrblobContent:img.jpg + $secret = Get-AzKeyVaultSecret - VaultName 'thevault' -Name 'SecretABCD' + $txt = $secret.SecretVaulue | ConvertFrom-SecureString -AsPlainText # readAKVsecret:plnTXT + AKV:[show_secret_value]:SecretABCD # Usage_VAULT_to_get_secret_talk_to_other_resource + if no managed identity at all = APPservice:local-option-storing-data + [APPservice]:Application_settings:name|value|source <> + < ${{secrets.Greeting}} # different value depending on environment + secret as part of Repo|ENV|Org >> not a Ton of secrets >> auth to the Cloud to access scrts + ()___pipeline___)====auth===[AKV]integration # AzureAPPregistr=github-devops:ServPrincipal + README.md : "clientID" # GitHub/secrets/REPOsecrets[^]:AzureCredentials(update) + creds: ${{ secrets.AZURE_CREDENTIALS }} # GitHubAction % $Secret ***hides even if nonNative + GITHUB_TOKEN = ${{ secrets.GITHUB_TOKEN }} # GITHUB Docs # what can be done w/GITHUBtoken ============================================================================================== [ > ]:::: DevOps Master Class Part 6 Infrastructure as Code ============================================================================================== + ()_deploy-infra__APP__Test__) # How do we normally do things + Imperative: what-to-do > tell cabdriver all steps to get to destination tell cabdriver to get to destination [IT-admin]--create--[RESOURCE] + {{[Policy]}}:guardrail:[IT-admin]:creates-limits(Security|Regulatory|Corp-Guidelines) + Azure-Policy:(Sec|Reg|CorpGdlns):ARM:portal|PS|CLI|Template + Terraform:[TF]:same-syntax -plan->[StateFile]--Apply-->[RESource] < multi|singleCloud + Azure-bicep: easier for humans to read ( vs ARM ) .bicep + parameters.json = VScode + VScode: PS> create-resource-portal > run-VScodePS > re-RunTempl:idempotent:sameResult + other-actions-not-in-template: deploymentScriptsComponents : more than 1 way to do it + ()_pipeline_)checkrepo_)auth_)deploy_) [.bicep+.json]repo + cred + env + [^]AzureCred:RepoLevel [^]Azure_SUB_ID:QA|test deployBicep.yml: Azure_SUB_ID.bicep.json + RunWorkFlow : auth...run..Failed?:policyViolation: PS|Manual|Portal|IaC + INFRASTRUCTURE: configuration:PS-DSC|CHeF|Puppet|Ansible:Declarative + IISinstall.ps1 : DSC -> what I want but not how to do it > version control + [IMG]~~~>[[PACKER]]--->[IMG]]]]shared-image-galley ]:::: Infrastructure as Code for the IT Administrator ============================================================================================== + IaC:imperativePS: New-AzResource-Group...New-AzStorageAccount...> works-for-simple-stuff + IaC:CI/CD:code.visualstudio.com:git-scm.com:github.com/powershell/powershell + StorageAccount.json : location . param . variables . resources . sku + VScode:enableSourceControl:"git -init":".git" [output]VScode : mainBranch : createMyBranch + git merge testnamechange > main-branch-now-has-the-change + dev.azure.com[Repos]IaC[v!Clone]https://lanid@dev.azure.com #newFolder>git clone @dev.azure + [dev.azure.com]releases:[+]NewReleasePipeline:EmptyJob:+AddArtifact:AzureRepo:IaC:masterBnch + [artifact][,+^]CDeplTrigg-->--{pre-depl-condit|appprvrs}[stages]+AddTask:AgentJob:deplARMtpl + [ADO][pipeline][task>AgentJob]:template_param:${sys.defWD/IaC/StorageAccount.parametrs.json + [+CreateRelease]:Stages:pulltasks:initialize>>Azure:storage_acc_created>ADO[Stage][Created!] + CDeploymentTrigger[enabled]>>VScode{modify.jsonfile}Save>SrcCTRL>[+stage][^commit-localonly] + "your branch is ahead of 'origin/master' by 1 commit" >VScode>[PUSH]>ADO>auto-deployTEMPLT + "depl-failed" > correct>stage>commit-locally>push>trigger-new-release # modifyCode ============================================================================================== [ > ]:::: Azure Master Class v2 - Module 11 - IaC & DevOps ============================================================================================== + [PC][portal:not-scalable-not-consistent]:ARM:((AzureCloud))<-azurePS:AZmoduleVScode[PSextns] + az login >browser>refreshAUTHtoken # AzureCloudShell-->((AzureCloud)){storage-account} + imperative is good for automation but not good for provisioning # ARG:AzureResourceGraph + ARM{RP}{RP}{ResrcProviders}((AZcloud))quota-ops:x-ms-ratelimit-remaining-subscription-reads + ((AZURE{read-only-DB:ARG})):requests-KQL:kusto- querying-language # ARGExplorer[query]reslt + Search-AzGraph Query '%|%|%' -Debug # ChangeAnalysis (AzurePortal) + ARG-rather-than-getAzVM|foreach>might-get-quota-limits>slow > AzureResourceGraph + DECLARATIVE > we-state-only-desired-end-state > easier-to-detect-drift<(AZ)(ARM)(TMPLbicep) + Declarative:DES:provisioning:detect-drift:version-control>>(Azure)< sameTEMPL + [TEMPLv2]<-prod.param|transfile|~~~>JSON(arm-json) # .bicep > microsoft.storage/storageAccount + bicep build # bicep decompile '%.json' >> .bicep file # ARM JSON > complex/noisy + files>Folder:[file2][file1]:snapshots:branches>git-scm.com/book # [__working_folder___] + [___WF___]}}}::..STAGE...::}}[__REPO__]: entireContentOfSnapshot ___/'"'\__ PR _merge_ + gitgraph ___/""""\___ # github repositories = github/ADO = priv/publ + {{ github-repo }} <>CODE|Local|Clone|HTTPS:github.com/lanid/repo123.git (1) + (2) Create + RemoteOrigin # DEVOPS: small.incremental.value = continuous-improvement + DevOPs: plan+track > develop.build.test.deploy.operate.monitor > plan+track + ()__pipeline__)..img-register..dev..build >> IaC ) APPcode ) load/testing >> ARM^((Azure)) + Azure GitHub Actions > AuthOIDC|secret > get-repo > deploy-template > GHactions>bicepDemo + Auth Using OIDC ... federatedTokenIssuer ... issuer:token.actions.githubusercontent.com ============================================================================================== [ > ]:::: IaC > infrastructure as code > ARM > Bicep > github > CI/CD/CD > ============================================================================================== + Azure verified modules [resource|utility|pattern]> Azure Verified Modules > Azure AVM team .bicep: module securestorage 'br:mynode.azurecr.io/bicep/private/pavm/ptm/secure-storage/vnet + CAF_cloud_adoption_frmwork:WAF_well-arch_frmwork:AVM_AzVerifiedModules:ALZ_AzLandingZones ---------------------------------------------------------------------------------------------- + resources.azure.com > Azure Resource Explorer : ARM JSON format > declarative/idempotent + 1 json template & 3 ENVs # ARM >> https://github.com/azure/azure-quickstart-templates + BICEP ---transpile_to---> ARM # [CLI] bicep decompile "$GitBasePath\storageAcount.json" + install Bicep CLI # Az CLI 2.20.0+ # PS Az 5.6.0. # bicep -v # az version # Bicep ext VS + intellisense+VSCode # name:'${mystorage2.name}/default/${containerName)' *~* "dependsOn":[] + New-AzResourceGroupDeployment -TemplateFile "$basePathGit\storageaccount.bicep" + Az deployment group create -f file.bicep -g RG --param param2=thevalue # whatif can be used + GitHub /Azure/bicep/docs : tutorials|examples 101 # create+consume modules # github repo ---------------------------------------------------------------------------------------------- ============================================================================================== [ > ]:::: ============================================================================================== ============================================================================================== [ + ] AZURE INFRASTRUCTURE ============================================================================================== |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ============================================================================================== [ > ]:::: ============================================================================================== ============================================================================================== [ > ]:::: ============================================================================================== ============================================================================================== [ > ]:::: ============================================================================================== ============================================================================================== [ + ] CYBERSEC ============================================================================================== |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ============================================================================================== [ > ]:::: Understanding and Getting Started with ZERO TRUST ============================================================================================== 1:verify explicitly 2:least privilege 3:assume breach(#)VPN (!)identity=user/app (SSO):MFA,passwordless,disable_legacy_auth (+)RBAC:JIT(PIM) (!)endpoint=TPM_trusted-platform-modules:device_cert+mutualTLS (+)register+managed:compliant (!)network=E2Eencryption:TLS/IPSEC (+)layers/tiers (+)microsegmentation:NSG+ASG+TLSinspection (!)network=collect signals:Azure Sentinel {+}CONTEXT((conditional_access)){+}CONTROL:MFA/session_control/block >grant_access:requireMFA:reqDEVICEcompliant:reqHybridAzureADjoined:reqApprvdClientAPP:pwdchg (!)INFRASTRUCTURE=trusted/limit_admins/JIT-JEA/mutualAUTH/signals/ [APP]:AzureAD-APP-proxy:VDI : sec_policy_at_service MScloud-app-security (proxyVDI) [APP]:find-shadow-IT:block_it (( DATA )): all roads lead to the data : Zero-trust : data driven protection : informProtect (( DATA )): inventory/classify*label///signals:bulk-copies (( DATA )): least-priv / data-masking / data-exfiltration / encryption: at-rest & in-use (( DATA )): SKU: encrypted in Memory : Azure purview : data criticality : probability << ZERO TRUST >> moving away from ISLAND/perimeter-only << IDENTITY >> << SIGNALS >> ML << SIEM >> security information and event management << SOAR >> security orchestration, automation, and response ============================================================================================== [ > ]:::: SC-100 Cybersecurity Architect Expert Certification Study Cram ============================================================================================== zero-trust:MCRA:microsoft_cybersecurity_reference_architectures # IDENTITY # IdentityProtection+risk(ca):P2_lic / MFA+CondAccess:P1_lic [ADDS[a]{adfs{a}:DefenderForIdentity--ADconnect(#of#):<>(IP-drkweb) [B2B]-<>-[B2C] # ENDPOINT # MEM:MSendpoint-mngr:intune+config-mngr MEM:CompliancePolicy <-> cond-access identity-protection:riskyUsers:riskySignIns >> compliance info fed into conditional access DFE: defender for endpoint : protect > detect > respond : unmanaged endpoints : cve:dashboard DFE <~~> MEM (CA) /// block behaviours and contents : antimalware : antivirus : automation DFS: defender for servers (NSG): adaptive APP hardening : FIM: file-integrity-monitoring # NETWORK # microsegmentation > VNET: [L4]NSG*5tuples+ASG UDR:AZFW APP_GW+WAF[L7]regional AzureFrontDoor:AFD[L7]global:TLS_offload:WAF:OWASPtop10 # DDoS prot # PrivateEndpoint(-)publE S2S/ERPP>>DNS>>privateEP # VNETintegration # APPserviceEnvironment # SQLmanaged-instance Kubernetes / Pods / NSG : granularity # INFRASTRUCTURE # DefenderForCloud:SecAlerts/cloudPosture/compliance/recomm/NIST/HIPAA/SWIFT # Azure Policy / BluePrint: Policy/RG/RBAC/template # ENCRYPTION: inTransit/atRest PMK:platformManagedKey/CMK:custManagedKey(rotate)/AKV:AzureKeyVault:Secret:key[hsm]:cert VMgen2[uefi:unifiedExtensibleFirmware^bios^]vTPM:virtualTrustedPlatformModule:trustedLaunch # Confidential Compute # Memory+CPU encrypt # OS / enclave # LEAST PRIVILEGE >> RBAC/JIT/AAD-PIM(P2lic)/AAD/ARM # PAM: privAccManagement <-> AD(on-prem) # Managed-identity:system_assigned/user_assigned RBAC:mngdID_cntrbRole/etc # Azure_Bastion: portal(ca)(JIT:ip/durati)[DFS] --> RDP/SSH: AzureVM # ARM: ARC---extend_other-clouds/on-prem azure-arc:server-K8s:data:app:ai # DFC: auto-agent-provisioning << defender for cloud >> policy # SIGNALS>CONTEXT:conditional-access[AADp1/p2risk]:CONTROL/block/grant # CONTROL >> APPS ((( AAD )))access_package(p2) {{SaaS}}defender-for-cloudAPPs # SaaS (reversePRXY)/session-cntrl : MDFCA: APPcatalog # DEVOPS >> GitHUB: Advanced-security:securityVuln-dependabots-secrets-pipeline-plainTXT # DATA >> MS-purview: discover/classify/lineage/label:IP/protection/SQLdataprotection # AzureBackup: PIN : MUA resource-guard # SIEM/SOAR : [Azure Sentinel]((LogAnalyticsWorkspace)):aad signals > INCident > Investigate > Respond: autom-rules # playbook : Azure Logic APP : visual representation ============================================================================================== [ > ]:::: Certificates of Authority Do you really understand how SSL TLS works ============================================================================================== [cert]: davidbombal.com |site-spoofing-test-research| david-bombal.com # crt.sh # domain-hist [letsencrypt.org]:free-tls-certificates:{https://certbot.eff.org} [cert]:certificate-chain+fields:command-line:[client]-<-[#]--[SERV]:[c]-_-_-[S]_-_-[CA]:trust [OS]:provided-with-root-certificates:transparent-for-the-user:cert-hierarchy:CA-ICA-{cert} [CA]-delegates_to_ICA-{cert}:signed-by-previous-certificate_Authority:padlock"trusted"forUser $ openssl s_client -connect theurl:443 # get /index.html host:theurl > ###: browserinteract --BEGIN-- --END-- # openssl x509 -in cert.crt -noout -text # human-readable-format:SAN:PubKey Subject:CN:presenting the certificate is not good enough to prove identity : privateKey=owner Modulus:RSA Public-Key (20248 bit): Asymmetric Key related to the Private-Key + 1000's users of computing time = Quantum Computing & Digital Crypto Security + AKI # SKI # : Hash of Private and Public Keys # SAN : Subject Alternative Name # DNS:domain + wildcard certificate: *.theurl.com does not protect theurl.com neither vm1.host2.theurl.com + KU & EKU: Only TLS web Client Auth # CRL distr-points: crl3.digicert : CRL-list:revokedCRTs + List of revoked certificates: badssl.com # revoked.badssl.com # wrong.host.badssl.com + openssl s_client -connect wrong.# open x509 # subject *.badssl.com : wrong.host.badssl! /\ + OCSP: Online Certificate Status Protocol : new way over CRL : CertPolicies: EV:extValidat + AIA:AuthorityInformatioAccess:OCSP+CAissuers(.crt) : X509constraints:>:CA:False + CT Precertificate SCT: Signed Certificate Timestamp : uploaded to public registry + Certificate Transparency = evaluates if anybody else is attempting to create certs w/yrName + Signature Algorithm:sha256WithRSAEncryption:to-sign-cert-SGNTR>BRWS>TRST>CA(RSAKEYencr-CRT) ============================================================================================== [ > ]:::: How SSL & TLS use Cryptographic tools to secure your data - Practical TLS ============================================================================================== + Confidentiality[encryption] [+] Integrity[hashing] [+] Authentication[PKI] + [hashing]:fingerprint+{MAC:msg-auth-code}integrity}authn} + [symm-encr]:encr+confdntlty [asymm-encr]:KeyExch{shrd-keys}+Sgntr{intg+authn}+Encr{confdnty} + [bulk-Data]:symm-encr(conf)+MsgAuthnCode(integ)<>AsymmEncrypt:[pubK][privK][CA ]:::: TLS Handshake Deep Dive and decryption with Wireshark ============================================================================================== + clientHello-> <-srvHelloChgCiphrSpec->fin-> <-ChgCiphrSpec <-FinEncr + confidentiality{symmEncr} [+] integrity{M.A.C.{hashing}} [+] authentication{PKI} + data-integr:[chgData?digest-on-modified-Data?]-->message-authentication-code-->secretKey + keys never cross the wire > digest can only be created by {secret-key} owner + HMAC : Hash based Message Authentication Code >openssl genrsa#asymm-keys # asymm-encr:hello # cannot-go-back # modulus-function-analogy + asymm = trap door functions # hello(key1)(key2)hello # privK+pubK = RSA = encr/decr + only the private key owner can decrypt what was encrypted with the corresponding public key + ciper-suite = [KeyExchange]+[Authentication]+[Encryption]+[Hashing] + [clientHello] = vers+rand#(timestamp)+SessionID+cipherSuites{supported}+Extension{+TLS/SSL} + [serverHello] = vers+rand#(bySRVR)+SessionID+cipherSuites{selected}+Extension{+TLS/SSL} + client+server = clear-TXT ! = both:session+cipher # wireshark filter by tls + WrShrk = Client Hello // server hello + certificate + server hello done + cipherSuite.e.g.: TLS_RSA_WITH_AES_128_CBC_SHA256 = keyXCHG+authn/encrypt/hashing + AES = block-ciphers|from-block-to-block > same-sentence-does-not-look-the-same if resent + server sends [cert]+[full-cert-chain]{publicKey} >>>[client] >>>clientKeyXCHG((SEED)) + clientKeyXCHG [Pre-Master-Secret]>>mutual-key-material #RSA|diffie-hellman + client>pre-master-secret>encryptedw/SRVRpubKey>[privKEY]>>>preMasterSecret"master secret" + (clientRND#+serverRND#)[MasterSecret]>>>"key expansion"(clientRND#+serverRND#)>>>4keys + SSL = generates 2 secure tunnels = client-encrK+HMACkey & server-encrK+HMACkey + server & client keys are different [SRV]:only-pair-of-key-can-decrypt:[CLNT] + client-key:encrypt&decrypt <--> server-key:encrypt&decrypt # client keys cannot decr srvr + both parties have the 4 keys available (the 2 pairs) : if someone brute-forces > 1 way only + 768bits (128x2+256x2) = 4 keys = CBC = client I.V. + server I.V. = PRF=PseudoRandomFunction + both have same starting point keys = the both end up with same key with same starting point + tell client to log session keys and use those session keys to decrypt conversation + WireShark = Client Key Exchange >> Encrypted Premaster ( needs server private key to decr ) + both parties have identical Session Key but do not know the other has the same Keys + they need to prove each other the have same identicaly keys : CHG cipher spec + [ pre-master-secret ] ---> [ encrypted-Verification ] 5:elements[handshake-Hash] + 5:elements[handshake-Hash]:client-hello/server-hello/certificate/server-hello-done/clientKE + [Handshake/"client finished"/MasterSecret]PRF-->[verificationData]:[encryptedVerification] + "server-knows-the-client-checks-out" --> client needs to know the server is the actual srvr + same process to confirm --> one more exchange --> encryptedVerification: (!)downgradeAttack + <<<< APPlication Data <<< sent to the server # WireShark : decrypt with Server Private Key + wireShark/TLSpacket/APPdata/ClientKeyExchange/ProtPref/RSAkeylist[edit]123.key[open][ok] + Encrypted Handshake Message >becomes> Finished > encryptedMSG > wireShark/followTLSstream + wireShark/followTLSstream >> human readable format > part of the 3rd message is encrypted + if private key is removed > Encrypted Handshake Message > WAF might-requires SRVR-privKey + Diffie+Hellman = cannot decrypt even with ServerPrivateKEY > forward secrecy property (PFS) ============================================================================================== [ > ]:::: ============================================================================================== ============================================================================================== SYSTEM DESIGN LEARNING NOTES ============================================================================================== |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ============================================================================================== non-relational-db = lowlatency.unstructured.JSON.XML.YAM.serialize.massive-amounts-of-data horizontal-scaling = large-scale-apps.maintenance.elastic-pool.no-outage vertical-scaling = load-limit.slow-response.conn-failure.outage.maintenance.load-balanced load-balancer = public-ip.isolate-servers.dmz.private-ip.failover.no-outage.sticky-session db-replication = master-slave.insert-delete-update:master-db.more-slaves-than-masters db-replication = better-performance.reliability.high-availability:locations:offline ---------------------------------------------------------------------------------------------- [USR]~~internet~~[web-server]:(r/w)[master-db]:::(r){slave-dbs} ---------------------------------------------------------------------------------------------- cache-layer||js:css:img:vid|| (-)load-response-time (+/-)expiration-policies ---------------------------------------------------------------------------------------------- cache-server = read-through-api.data-type.data-size.access-pattern.frequent-read.infreq-write cache-server-consistency = multi-region-scaling.mitigate-failures.spof.eviction-policy cache-eviction-policy = lru:least-recently-used.lfu:least-frequently-used.fifo:1st-in-1st-out cdn = if-not-in-cdn:[origin-server]~~{http-header+ttl.time-to-live}:data-transfer.in-out.cost cdn-fallback | client-request-content-to-origin | invalidating-files-via-api | obj-versioning ---------------------------------------------------------------------------------------------- stateless-web-tier = user-sessions.store-in-relational-db = usr-session-not-in-web-tier ---------------------------------------------------------------------------------------------- stateful-arch = sticky-sessions(((overhead))) stateless-arch = ~~(fetch-usr-state-data) ---------------------------------------------------------------------------------------------- GeoDNS = (traffic)~~[nearest-data-center]::data-sync::local-db.failover.replicated-datacenter message-queuing = scalable-web-servers:::message-queue[m][m][m]:::scalable-backend monitoring = scalable-logging + scalable-metrics + automation.ci/cd/cd ---------------------------------------------------------------------------------------------- + stateless-web-tier + tier-redundancy + cache-layer + multi-datacenter:UD:FD:Zones:Regions + cdn-static-obj-hosting + scale-data-tier:sharding + tiered-services + monitoring+automation ============================================================================================== NIST = 5-characteristics.3-service-models.4-deployment-models NIST = on-demand-self-serv:broad-netw-access:resource-pooling:rapid-elasticity.measured-srvc NIST = SaaS = app-config-settings.info-management.use-only.no-INF-MNGMT.no-INF-CTRL NIST = PaaS = deploy-consumer/vendor-apps-code.some-INF-SETTINGS-MNGMT.no-INF-CTRL NIST = IaaS = virtual-INF-creation.VMs.OS.FW.WAF.PRXY.NETW.deploy-consumer/vendor-apps-code NIST = private-cloud = company-owned-managed-operated/3rd-party.on-premise/off-premise NIST = community-cloud = shared-concerns:mission-security-compliance.on-premise/off-premise NIST = public-cloud = on-cloud-provider-premises = ownd-mngd-opr-by-business/academic-org/gov NIST = hybrid-cloud = two-or-more-separate-distinct-cloud-inf:priv/public:cloud-app-LB/access ---------------------------------------------------------------------------------------------- SaaS = [table.drink.gas.oven.fire.cheese.tomato.dough]:vendor_mngmt PaaS = [table.drink]:customer_mngmt::[gas.oven.fire.cheese.tomato.dough]:vendor_mngmt IaaS = [table.drink.gas.oven.fire]:customer_mngmt::[cheese.tomato.dough]:vendor_mngmt PRIV = [table.drink.gas.oven.fire.cheese.tomato.dough]:customer_mngmt ---------------------------------------------------------------------------------------------- SaaS = [app.data.runtime.middleware.OS.VIRT.SERV.STO.NET]:vendor_mngmt PaaS = [app.data]:customer_mngmt::[runtime.middleware.OS.VIRT.SERV.STO.NET]:vendor_mngmt IaaS = [app.data.runtime.middleware.OS]:customer_mngmt::[VIRT.SERV.STO.NET]:vendor_mngmt PRIV = [app.data.runtime.middleware.OS.VIRT.SERV.STO.NET]:customer_mngmt ---------------------------------------------------------------------------------------------- + Design for self healing + Make all things redundant + Minimize coordination + Partition around limits + Design for operations + Use managed services + Use an identity service + Design for evolution + Build for the needs of business ==============================================================================================