GEA submission flow
1. Obtain a submission account
- Create a D-way submission account.
- To enable GEA submission, register a public key and a center name to your account.
2. Register a BioProject, BioSamples and Sequence Read Archive (DRA)
BioProject and BioSample registration is required for DRA submission.
- A description of the reseach effort.
- A description of biologically or physically unique samples used to generate experimental data.
- Raw sequencing reads and alignments.
metadata can be submitted as a tab-delimited text file.
3. Upload processed data files
- Upload processed data files into the GEA submission directory.
4. Select a DRA submission
- Select a DRA submission which contains raw sequencing reads for the GEA submission.
- Select DRA submissions when the GEA experiment has processed data from raw sequencing reads in multiple DRA submissions.
5. Select a BioProject
- Select a BioProject used in the DRA submission when the GEA experiment and DRA submission belong to same project.
- Select a BioProject not used in the DRA submission when the GEA experiment and DRA submission belong to different projects.
6. Prepare IDF and SDRF
- The IDF (Investigation Description Format) file is used to give an overview of the experiment, including the experimental design, protocols and publication information.
- The SDRF (Sample and Data Relationship Format) describes the sample characteristics and the relationship between samples, data files etc.
- SDRF template is generated from selected BioProject, BioSample(s) and DRA submission(s). Enter additional information.
7. Submit IDF and SDRF and validate data files
- After submitting IDF and SDRF metadata in the submission web system, validation of the uploaded data files are automatically started.
- The submission passed validation will be reviewed.
GEA will ask you for the technology and name of the array, and applies it to the whole submission. If you have used different types of technologies for the same set of samples, we ask you to create separate submissions. Please make sure that the submissions have distinct titles (even though they may belong to the same study), in order to avoid mistakes. If you have samples from more than one array design in your experiment, it is possible to submit only one experiment. If you wish to do this, please contact GEA team.
Sequencing experiment submission
Create a new submission
Login the D-way (https://trace.ddbj.nig.ac.jp/D-way) and the top page is displayed. Move to the GEA submission site from the “GEA” menu at the top.
Create a new microarray experiment submission by selecting “Sequencing” and clicking the [New submission]. At the same time, in the DDBJ file server (ftp-private.ddbj.nig.ac.jp), a corresponding subdirectory is created under the submitter’s GEA upload directory. Upload data files to this subdirectory.
If there is no reply from submitters after three months of initial contact, submissions will be cancelled.
The maximum number of assay per submission is 1,000. If you have more than 1,000 assays, please create multiple submissions with the same BioProject reference.
List of submission status is as follows. The GEA team reviews submission whose status is in “submission_validated” or “data_error”.
|New||Metadata are not submitted.|
|Data Submitted||Metadata and data files are submitted.|
|Data Validating||Validating data files.|
|Validation Error||Error occurred in data validation process.|
|Data Validated||Metadata and data are validated.|
|Curating||Curator is reviewing the submission.|
|Accession Issued||Accession number is issued to the submission.|
|Confidential||Archive files are created and submission is kept private|
|Public||Released to public.|
Upload processed data files
Upload files by executing,
- <Your Files> Files to be transferred. Ex: file1 file2 (file1 and file2), file* (all files whose filenames start with “file”)
- <D-way Login ID> D-way Login ID (ex. test07)
- <GEA Submission ID> GEA Submission ID (ex. ESUB000350)
- command example: scp Arabidopsis_control_rep_1.CEL firstname.lastname@example.org:~/gea/ESUB000350
Enter the passphrase set for the keys.
You can directly handle the transferred files by logging in the server. SSH login the server by executing,
Enter the passphrase set for the keys.
After logging in successfully, the following prompt is displayed.
The login environment is private for the submitter. Users other than the submitter cannot access the data. Executable commands are restricted to the following ones. Users can delete unnecessary files.
Submission to DRA ～upload data files (Windows)～
Install and run the “WinSCP” (http://winscp.net/eng/download.php).
Set items as below and click the [Advanced…] button.
Be sure to select the “binary mode” for file transfer. Do NOT select the “text mode”.
- File protocol: SFTP
- Host name: ftp-private.ddbj.nig.ac.jp
- Port number: 22
- User name: (D-way Login ID)
- Password: (Leave empty)
Please select the private key, which you created beforehand, from “Private key file” in “Authentication”.
Last, click the [Login] button in the lower center
At the first time of login, a warning message is displayed; however, please select “Yes” (this message will not be displayed again). Next, enter the passphrase set for the keys.
After login successfully, a folder of your PC is displayed at left, and your private directory in the server is displayed at right. Select the files at the left window and drag & drop them into the right window to transfer the files to the server.
You can delete the transferred files by selecting the files and clicking the [Delete] button.
Submission to DRA ～upload data files (Mac) ～
Download and install the Cyberduck (http://cyberduck.ch).
Run the Cyberduck and click the [Open Connection] button in the Cyberduck menu.
Select “SFTP (SSH File Transfer Protocol)” .
Set as follows and tick off “Use Public Key Authentication” in the More Options.
- Server: ftp-private.ddbj.nig.ac.jp
- Port: 22
- Username: (D-way Login ID)
- Password: (Leave empty)
- Add to Keychain: (Check)
By default, the private key is created in “User’s home folder > .ssh folder (invisible in Finder) > id_rsa”.
At the first time of login, a warning message is displayed; however, please select “Always” (this message will not be displayed again).
After login successfully, your private directory in the server is displayed in the window. Select the files in your PC and drag & drop them into the window to transfer the files to the server.
Users can ssh login ftp-private.ddbj.nig.ac.jp server by using a private key. Executable commands are restricted to the following ones. Users can delete unnecessary files.
ls cd cp mv rm more mkdir tar gzip gunzip bzip2 bunzip2 zip unzip
Set the hold date within four years. Submitters’ name and affiliation will be public but e-mail address will not be disclosed.
You can delete an un-submitted GEA submission.
Select a DRA submission registered in your account. If DRA submission is not registered, please go to the DRA submission site and submit DRA.
To use DRA submissions obtained in the other account, please contact GEA team.
Select a project registered in your account. If a BioProject is not registered, please go to the BioProject submission site and submit a project.
Select a BioProject used in the DRA submission when the GEA experiment and DRA submission belong to same project (usual case). When the GEA experiment and DRA submission belong to different projects, select a BioProject not used in the DRA submission.
To use a project obtained in the other account, please contact GEA team.
Enter information for IDF (Investigation Description Format).
- Protocol: Pre-checked protocols are mandatory.
- Publication: Describe associated publications by PubMed ID or DOI. For unpublished manuscript, please inform us the ID after assignment.
- Data File Type: Processed data files are required for sequencing experiment submission. Accepted Data Files Formats for sequencing experiment. We strongly recommend to submitting processed data file per sample.
Enter information for SDRF (Sample and Data Relationship Format).
- Name columns and attribute columns for Source Name: Generated from BioSamples.
- SDRF rows: 1 row for 1 Run.
- Protocols: Protocols described in IDF are inserted to appropriate positions of SDRF with temporary protocol IDs (e.g., ESUB000352_Protocol_1)
- Technology Type: “sequencing assay” for sequencing submission.
- SRA Experiment and Run Comments to Extract and Assay Names: Generated from DRA Experiment and Run.
Enter required fields by overwriting <Required: fill in the content> tags.
Fields you need to add.
- Material Type: Enter controlled terms.
- total RNA
- polyA RNA
- cytoplasmic RNA
- nuclear RNA
- genomic DNA
- Derived Array Data File and Comment[Derived Array Data File md5]: Enter filename and md5 checksum pair for each processed data file.
- A list of filename and its md5 checksum (output of md5sum command) can be provided as a file <GEA submission ID>.md5 (e.g., ESUB000001.md5) (when the checksum values are provided in both SDRF and .md5 file, those in the .md5 are used).
- Factor Value[enter experiment factor name here]: A user-defined name for each experimental factor studied by the experiment. These experimental factors represent the variables within the investigation (e.g. growth condition, genotype, organism part). The actual values of these variables will be listed in the “Factor Value ” columns. Example:
- Factor Value[strain]
Select the entered SDRF file and continue.
Overview and submit
You can download the IDF and SDRF files and check them. When correction is necessary, go back to the previous tab and corrent metadata.
Submit the IDF and SDRF metadata by clicking the “Submit” button.
When data files described in the IDF and SDRF are not found in the submission directory, an error message “Data file is not uploaded” is shown and the submission is aborted.
The validator checks submitted IDF and SDRF files according to the validation rules. and gives warning and error messages. Errors need to be resolved for submission.
Update in each database
|Annotated sequence database||Request updates from web form|
|Sequence Read Archive (DRA)||Login D-way and update metadata
To add or withdraw sequencing data, request updates from web form
|Genomic Expression Archive (GEA)||Request updates from web form|
|BioProject/BioSample||Request updates from web form|
Withdraw archived objects
To withdrawing archived Experiment, please contact us.
MD5 (Message Digest Algorithm 5) is a hash function which calculates a hash value (MD5 number, 32-digit numbers and letters) of a given file. Because the MD5 number of the damaged file is distinct from the original one, we can check whether the transferred file is intact or not by comparing the numbers before and after the file transfer.
Obtain the MD5 numbers of the files by executing,
$ md5sum file1 file2 9F6E6800CFAE7749EB6C486619254B9C file1 B636E0063E29709B6082F324C76D0911 file2
Obtain the MD5 numbers of the files by executing,
$ md5 file1 file2 9F6E6800CFAE7749EB6C486619254B9C file1 B636E0063E29709B6082F324C76D0911 file2
Install and run the Fsum Frontend (sourceforge.net/projects/fsumfe/).
At first, tick off “md5”.
After clicking the [+] button, open the sequence data files that you need. You can select multiple files at the same time.
Click the [Calculate hashes] button. The MD5 numbers of the files are displayed.
By clicking the [Export] button, you can obtain the list of the MD5 numbers as a html, a csv, or a xml file.