1. 首頁
  2. 問答
  3. 如何處理CodePipeline中的多個環境？

如何處理CodePipeline中的多個環境？

2017-01-16 80 views
2

我正在使用代碼管道來部署我的基礎架構，我希望能夠在不同的環境（dev，staging，prod，...）中部署它。如何處理CodePipeline中的多個環境？

我現在有一些含有「PIP安裝」的說明和「AWS cloudformation包」命令buildspec.yml文件。我還創建了2個管道，一個用於生產，另一個用於開發，指向github上的2個不同分支。我遇到的問題是，因爲在兩個分支中文件都包含相似的資源，所以我在S3存儲桶上存在名稱衝突。

在使用AWS CLI和cloudformation創建或更新一個堆棧，你可以通過使用--parameters選項參數。我想在我創建的2個管道中做類似的事情。

什麼是解決這一問題的最佳解決方案？

的最終目標是自動化我們的基礎設施的部署。我們的基礎架構由用戶，KMS密鑰，Lamdbas（在Python中），組和存儲桶組成。

我已創建的教程以下兩條管線：http://docs.aws.amazon.com/lambda/latest/dg/automating-deployment.html

第一管道被連接到包含代碼，第二個到臨時分支回購的主分支。我的目標是使用第二個管道環境中的第一個管道和臨時分支在臨時環境中自動部署生產環境中的主分支。

我buildspec.yml文件的樣子：

version: 0.1 
phases: 
    install: 
     commands: 
      - pip install requests -t . 
      - pip install simplejson -t . 
      - pip install Image -t . 
      - aws cloudformation package --template-file image_processing_sam.yml --s3-bucket package-bucket --output-template-file new_image_processing_sam.yml 
artifacts: 
    type: zip 
    files: 
     - new_image_processing_sam.yml

的image_processing_sam.yml文件的樣子：

AWSTemplateFormatVersion: "2010-09-09" 
Transform: "AWS::Serverless-2016-10-31" 
Description: Create a thumbnail for an image uploaded to S3 
Resources: 

    ThumbnailFunction: 
    Type: "AWS::Serverless::Function" 
    Properties: 
     Role: !GetAtt LambdaExecutionRole.Arn 
     Handler: create_thumbnail.handler 
     Runtime: python2.7 
     Timeout: 30 
     Description: "A function computing the thumbnail for an image." 

    LambdaSecretEncryptionKey: 
    Type: "AWS::KMS::Key" 
    Properties: 
     Description: "A key used to encrypt secrets used in the Lambda functions" 
     Enabled: True 
     EnableKeyRotation: False 
     KeyPolicy: 
     Version: "2012-10-17" 
     Id: "lambda-secret-encryption-key" 
     Statement: 
      - 
      Sid: "Allow administration of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role" 
      Action: 
       - "kms:Create*" 
       - "kms:Describe*" 
       - "kms:Enable*" 
       - "kms:List*" 
       - "kms:Put*" 
       - "kms:Update*" 
       - "kms:Revoke*" 
       - "kms:Disable*" 
       - "kms:Get*" 
       - "kms:Delete*" 
       - "kms:ScheduleKeyDeletion" 
       - "kms:CancelKeyDeletion" 
      Resource: "*" 
      - 
      Sid: "Allow use of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: 
       - !GetAtt LambdaExecutionRole.Arn 
      Action: 
       - "kms:Encrypt" 
       - "kms:Decrypt" 
       - "kms:ReEncrypt*" 
       - "kms:GenerateDataKey*" 
       - "kms:DescribeKey" 
      Resource: "*" 

    LambdaExecutionRole: 
    Type: "AWS::IAM::Role" 
    Properties: 
     RoleName: "LambdaExecutionRole" 
     AssumeRolePolicyDocument: 
     Version: "2012-10-17" 
     Statement: 
     - Effect: Allow 
      Principal: 
      Service: 
      - "lambda.amazonaws.com" 
      Action: 
      - "sts:AssumeRole" 
     Policies: 
     - 
      PolicyName: LambdaKMS 
      PolicyDocument: 
      Version: "2012-10-17" 
      Statement: 
       - 
       Effect: "Allow" 
       Action: 
        - "kms:Decrypt" 
       Resource: "*" 
       - 
       Effect: "Allow" 
       Action: 
        - "lambda:InvokeFunction" 
       Resource: "*" 
     ManagedPolicyArns: 
     - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" 

    UserGroup: 
     Type: "AWS::IAM::Group" 

    LambdaTriggerUser: 
    Type: "AWS::IAM::User" 
    Properties: 
     UserName: "LambdaTriggerUser" 

    LambdaTriggerUserKeys: 
    Type: "AWS::IAM::AccessKey" 
    Properties: 
     UserName: 
     Ref: LambdaTriggerUser 

    Users: 
    Type: "AWS::IAM::UserToGroupAddition" 
    Properties: 
     GroupName: 
     Ref: UserGroup 
     Users: 
     - Ref: LambdaTriggerUser 

    Policies: 
    Type: "AWS::IAM::Policy" 
    Properties: 
     PolicyName: UserPolicy 
     PolicyDocument: 
     Statement: 
      - 
      Effect: "Allow" 
      Action: 
       - "lambda:InvokeFunction" 
      Resource: 
       - !GetAtt DispatcherFunction.Arn 
     Groups: 
     - Ref: UserGroup 

    PackageBucket: 
    Type: "AWS::S3::Bucket" 
    Properties: 
     BucketName: "package-bucket" 
     VersioningConfiguration: 
     Status: "Enabled" 

Outputs: 
    LambdaTriggerUserAccessKey: 
    Value: 
     Ref: "LambdaTriggerUserKeys" 
    Description: "AWSAccessKeyId of LambdaTriggerUser" 

    LambdaTriggerUserSecretKey: 
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey 
    Description: "AWSSecretKey of LambdaTriggerUser"

我已在兩條流水線部署動作來執行計算的變更集在測試行動期間。

第一管線工程就像一個魅力和做的一切我期望它做的事。每次我在主分支中推送代碼時，都會部署它。

我現在面臨的問題是，當我在臨時分支推代碼，一切都在流水線工作，直到達到部署行動。部署操作嘗試創建一個新的堆棧，但由於它與處理的buildspec.yml和image_processing_sam.yml完全相同，因此我會遇到下面的名稱衝突。

package-bucket already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx 
LambdaTriggerUser already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx 
LambdaExecutionRole already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx 
...

有沒有辦法參數化buildspec.yml能夠添加一個後綴在image_processing_sam.yml資源的名字嗎？任何其他想法實現這一點是值得歡迎的。

此致敬禮。

來源

2017-01-16 JonathanGailliez

回答

2

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html

模板配置文件通過參數文件應用到CodePipeline的CloudFormation這樣的：

{ 
    "Parameters" : { 
    "DBName" : "TestWordPressDB", 
    "DBPassword" : "TestDBRootPassword", 
    "DBRootPassword" : "TestDBRootPassword", 
    "DBUser" : "TestDBuser",  
    "KeyName" : "TestEC2KeyName" 
    } 
}

將這些文件在您的回購協議的根源，他們可以被引用在至少有兩種方法。

在你CodePipeline CloudFormation：

Configuration: 
    ActionMode: REPLACE_ON_FAILURE 
    RoleArn: !GetAtt [CFNRole, Arn] 
    StackName: !Ref TestStackName 
    TemplateConfiguration: !Sub "TemplateSource::${TestStackConfig}" 
    TemplatePath: !Sub "TemplateSource::${TemplateFileName}"

或者在模板配置字段控制檯： enter image description here

值得注意的是，該配置文件的格式是通過CLI從CloudFormation不同的使用

-- parameters

- 參數使用此格式：

[ 
    { 
    "ParameterKey": "team", 
    "ParameterValue": "AD-Student Life Applications" 
    }, 
    { 
    "ParameterKey": "env", 
    "ParameterValue": "dev" 
    }, 
    { 
    "ParameterKey": "dataSensitivity", 
    "ParameterValue": "public" 
    }, 
    { 
    "ParameterKey": "app", 
    "ParameterValue": "events-list-test" 
    } 
]

CodePipeline Cloudformation模板配置文件，請使用以下格式：

{ 
    "Parameters" : { 
    "DBName" : "TestWordPressDB", 
    "DBPassword" : "TestDBRootPassword", 
    "DBRootPassword" : "TestDBRootPassword", 
    "DBUser" : "TestDBuser",  
    "KeyName" : "TestEC2KeyName" 
    } 
}

來源

2017-01-19 17:22:29

+1

您好，我沒有這個時間這個星期來測試，但我一定要嘗試一下，儘快接受你的答案。感謝您的時間和詳細的答案。 – JonathanGailliez

+0

昨天測試完整代碼示例可在此處獲得第271行： https://github.com/byu-oit-appdev/iac/blob/master/cloudformation/codepipeline/lambda-pipeline-cf.yaml –

+1

嗨，Eric，我終於設法花時間嘗試和驗證您的解決方案。這確實是繼續下去的方式。再次感謝您的詳細解答和您的時間。最好的祝福。 – JonathanGailliez

1

檢查Eric Nord的答案。這是你正在尋找的人。

我問了AWS論壇上的問題以及here。

這裏是由AWS提供的解決方案：

嗨，

如果你的目標是有分期和掌握不同桶的名字，然後又選擇是使用CloudFormation參數。

編輯現有管道時，如果編輯操作，則可以展開「高級」面板並輸入參數覆蓋以爲每個階段指定不同的存儲桶前綴。您還可以在工件中將參數作爲單獨的.json文件輸入。

有做的是，這裏更多的細節：http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-parameter-override-functions.html

以下是帶有測試和生產不同的堆棧配置的全穿行：http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html

  • 蒂姆。

你一定要按照提供的文件。以下是我在下面提出的解決方案。

這是我自己的解決方案，我不滿意。

我已經添加了一個在構建時運行的腳本，並修改了我的模板，給出了構建項目的CodeBuild代理的ARN。

我添加了「BRANCH_NAME」，其中可能會發生命名衝突。該image_processing_sam.yml現在是：

AWSTemplateFormatVersion: "2010-09-09" 
Transform: "AWS::Serverless-2016-10-31" 
Description: Create a thumbnail for an image uploaded to S3 
Resources: 

    ThumbnailFunction: 
    Type: "AWS::Serverless::Function" 
    Properties: 
     Role: !GetAtt LambdaExecutionRole.Arn 
     Handler: create_thumbnail.handler 
     Runtime: python2.7 
     Timeout: 30 
     Description: "A function computing the thumbnail for an image." 

    LambdaSecretEncryptionKey: 
    Type: "AWS::KMS::Key" 
    Properties: 
     Description: "A key used to encrypt secrets used in the Lambda functions" 
     Enabled: True 
     EnableKeyRotation: False 
     KeyPolicy: 
     Version: "2012-10-17" 
     Id: "lambda-secret-encryption-keyBRANCH_NAME" 
     Statement: 
      - 
      Sid: "Allow administration of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role" 
      Action: 
       - "kms:Create*" 
       - "kms:Describe*" 
       - "kms:Enable*" 
       - "kms:List*" 
       - "kms:Put*" 
       - "kms:Update*" 
       - "kms:Revoke*" 
       - "kms:Disable*" 
       - "kms:Get*" 
       - "kms:Delete*" 
       - "kms:ScheduleKeyDeletion" 
       - "kms:CancelKeyDeletion" 
      Resource: "*" 
      - 
      Sid: "Allow use of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: 
       - !GetAtt LambdaExecutionRole.Arn 
      Action: 
       - "kms:Encrypt" 
       - "kms:Decrypt" 
       - "kms:ReEncrypt*" 
       - "kms:GenerateDataKey*" 
       - "kms:DescribeKey" 
      Resource: "*" 

    LambdaExecutionRole: 
    Type: "AWS::IAM::Role" 
    Properties: 
     RoleName: "LambdaExecutionRoleBRANCH_NAME" 
     AssumeRolePolicyDocument: 
     Version: "2012-10-17" 
     Statement: 
     - Effect: Allow 
      Principal: 
      Service: 
      - "lambda.amazonaws.com" 
      Action: 
      - "sts:AssumeRole" 
     Policies: 
     - 
      PolicyName: LambdaKMSBRANCH_NAME 
      PolicyDocument: 
      Version: "2012-10-17" 
      Statement: 
       - 
       Effect: "Allow" 
       Action: 
        - "kms:Decrypt" 
       Resource: "*" 
       - 
       Effect: "Allow" 
       Action: 
        - "lambda:InvokeFunction" 
       Resource: "*" 
     ManagedPolicyArns: 
     - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" 

    UserGroup: 
     Type: "AWS::IAM::Group" 

    LambdaTriggerUser: 
    Type: "AWS::IAM::User" 
    Properties: 
     UserName: "LambdaTriggerUserBRANCH_NAME" 

    LambdaTriggerUserKeys: 
    Type: "AWS::IAM::AccessKey" 
    Properties: 
     UserName: 
     Ref: LambdaTriggerUser 

    Users: 
    Type: "AWS::IAM::UserToGroupAddition" 
    Properties: 
     GroupName: 
     Ref: UserGroup 
     Users: 
     - Ref: LambdaTriggerUser 

    Policies: 
    Type: "AWS::IAM::Policy" 
    Properties: 
     PolicyName: UserPolicyBRANCH_NAME 
     PolicyDocument: 
     Statement: 
      - 
      Effect: "Allow" 
      Action: 
       - "lambda:InvokeFunction" 
      Resource: 
       - !GetAtt DispatcherFunction.Arn 
     Groups: 
     - Ref: UserGroup 

    PackageBucket: 
    Type: "AWS::S3::Bucket" 
    Properties: 
     BucketName: "package-bucketBRANCH_NAME" 
     VersioningConfiguration: 
     Status: "Enabled" 

Outputs: 
    LambdaTriggerUserAccessKey: 
    Value: 
     Ref: "LambdaTriggerUserKeys" 
    Description: "AWSAccessKeyId of LambdaTriggerUser" 

    LambdaTriggerUserSecretKey: 
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey 
    Description: "AWSSecretKey of LambdaTriggerUser"

的script.sh模板替換 「BRANCH_NAME」 是：

#!/bin/bash 
echo $CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN 
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"master"* ]]; then 
    sed "s/BRANCH_NAME//g" image_processing_sam.yml > generated_image_processing_sam.yml; 
fi 
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"staging"* ]]; then 
    sed "s/BRANCH_NAME/staging/g" image_processing_sam.yml > generated_image_processing_sam.yml; 
fi

的buildspec.yml現在是：

version: 0.1 
phases: 
    install: 
     commands: 
      # Install required module for python 
      - pip install requests -t . 
      - pip install simplejson -t . 
      - pip install Image -t . 
      - bash ./script.sh 
      # To be able to see any issue in the generated template 
      - cat generated_image_processing_sam.yml 
      # Package the generated cloudformation template in order to deploy 
      - aws cloudformation package --template-file generated_image_processing_sam.yml --s3-bucket piximate-package-bucket --output-template-file new_image_processing_sam.yml 
artifacts: 
    type: zip 
    files: 
     - new_image_processing_sam.yml

我希望它能以某種方式幫助你。如果有人能提供任何可以幫助的改進或文檔，我會很高興。

來源

2017-01-16 15:30:51 JonathanGailliez

相關問題

 相關問題