Train S3 location from ExecutionInput

[sumeshmr]

This is an excellent tool, great Work!. I was looking forward to future releases.

I was trying the below example:
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/step-functions-data-science-sdk/automate_model_retraining_workflow/automate_model_retraining_workflow.ipynb

I have a requirement to rerun the same workflow each time with a different s3 location.

training_step = steps.TrainingStep( 'Model Training', estimator=xgb, data={ 'train': s3_input(train_data, content_type='csv'), 'validation': s3_input(validation_data, content_type='csv') }, job_name=execution_input['TrainingJobName'], wait_for_completion=True )

In this example, the train and validation s3 locations are fixed. I want to change this on each workflow execution. Is there a way to input this from ExecutionInput or any other means?

[ParidelPooya]

Hi,

Thank you for bringing it to our attention. Accepting s3 address as ExecutionInput is a valid use case and we will look in to addressing it in StepFunction SDK.

Pooya

[sree-kanth]

On similar lines, I want to specify the FileSystemDataSource, so that the data picked up from S3Uri gets written to the /opt/ml/input/data/training folder of the docker container. How do I specify that in the Training_Step?

[ca-nguyen]

With #174, using a different input s3 location will be possible by using placeholders such as ExecutionInput to define the InputDataConfig in the parameters argument.

This will be available with next release