GEA submission flow
1. Obtain a submission account
- Create a D-way submission account.
- To enable GEA submission, register a public key and a center name to your account.
2. Register a BioProject and BioSample(s)
- A description of the reseach effort.
- A description of biologically or physically unique samples used to generate experimental data.
metadata can be submitted as a tab-delimited text file.
3. Upload raw and processed data files
- Upload raw and processed data files into the GEA submission directory.
- [Optional] When an array design is not available in ArrayExpress/GEA, upload an array design file into the GEA submission directory along with data files.
4. Select a BioProject and BioSample(s)
- Select a registed BioProject for the GEA submission.
- Select registed BioSample(s) for the GEA submission. Majority of GEA submissions require more than one sample.
5. Prepare IDF and SDRF
- The IDF (Investigation Description Format) file is used to give an overview of the experiment, including the experimental design, protocols and publication information.
- The SDRF (Sample and Data Relationship Format) describes the sample characteristics and the relationship between samples, arrays, data files etc.
- SDRF template is generated from selected BioProject and BioSample(s). Enter additional information.
6. Submit IDF, SDRF and validate data files
- After submitting IDF and SDRF in the submission web system, validation of the uploaded data files will automatically begin.
- The submission passed validation will be reviewed.
At the moment GEA submission interface only supports one type of two-color workflow (see graphic here), where two samples are connected with one common raw data file, which includes both channels. If you select the dual-channel option in the IDF tab, it will expect that you provide one file for the two samples that were hybridized together. Some recent two-color microarray technologies generate two separate raw data files (usually one for each channel), which will cause validation to fail (if you connect a single file per sample).
If you have separate files for each channel, please contact GEA team.
GEA will ask you for the technology and name of the array, and applies it to the whole submission. If you have used different types of technologies for the same set of samples, we ask you to create separate submissions. Please make sure that the submissions have distinct titles (even though they may belong to the same study), in order to avoid mistakes. If you have samples from more than one array design in your experiment, it is possible to submit only one experiment. If you wish to do this, please contact GEA team.
Microarray experiment submission
Create a new submission
Login the D-way (https://trace.ddbj.nig.ac.jp/D-way) and the top page is displayed. Move to the GEA submission site from the “GEA” menu at the top.
Create a new microarray experiment submission by selecting “Microarray” and clicking the [New submission]. At the same time, in the DDBJ file server (ftp-private.ddbj.nig.ac.jp), a corresponding subdirectory is created under the submitter’s GEA upload directory. Upload data files to this subdirectory.
If there is no reply from submitters after three months of initial contact, submissions will be cancelled.
The maximum number of assay per submission is 1,000. If you have more than 1,000 assays, please create multiple submissions with the same BioProject reference.
List of submission status is as follows. The GEA team reviews submission whose status is in “submission_validated” or “data_error”.
|New||Metadata are not submitted.|
|Data Submitted||Metadata and data files are submitted.|
|Data Validating||Validating data files.|
|Validation Error||Error occurred in data validation process.|
|Data Validated||Metadata and data are validated.|
|Curating||Curator is reviewing the submission.|
|Accession Issued||Accession number is issued to the submission.|
|Confidential||Archive files are created and submission is kept private|
|Public||Released to public.|
Upload raw and processed data files
Upload files by executing,
- <Your Files> Files to be transferred. Ex: file1 file2 (file1 and file2), file* (all files whose filenames start with “file”)
- <D-way Login ID> D-way Login ID (ex. test07)
- <GEA Submission ID> GEA Submission ID (ex. ESUB000350)
- command example: scp Arabidopsis_control_rep_1.CEL email@example.com:~/gea/ESUB000350
Enter the passphrase set for the keys.
You can directly handle the transferred files by logging in the server. SSH login the server by executing,
Enter the passphrase set for the keys.
After logging in successfully, the following prompt is displayed.
The login environment is private for the submitter. Users other than the submitter cannot access the data. Executable commands are restricted to the following ones. Users can delete unnecessary files.
Submission to DRA ～upload data files (Windows)～
Install and run the “WinSCP” (http://winscp.net/eng/download.php).
Set items as below and click the [Advanced…] button.
Be sure to select the “binary mode” for file transfer. Do NOT select the “text mode”.
- File protocol: SFTP
- Host name: ftp-private.ddbj.nig.ac.jp
- Port number: 22
- User name: (D-way Login ID)
- Password: (Leave empty)
Please select the private key, which you created beforehand, from “Private key file” in “Authentication”.
Last, click the [Login] button in the lower center
At the first time of login, a warning message is displayed; however, please select “Yes” (this message will not be displayed again). Next, enter the passphrase set for the keys.
After login successfully, a folder of your PC is displayed at left, and your private directory in the server is displayed at right. Select the files at the left window and drag & drop them into the right window to transfer the files to the server.
You can delete the transferred files by selecting the files and clicking the [Delete] button.
Submission to DRA ～upload data files (Mac) ～
Download and install the Cyberduck (http://cyberduck.ch).
Run the Cyberduck and click the [Open Connection] button in the Cyberduck menu.
Select “SFTP (SSH File Transfer Protocol)” .
Set as follows and tick off “Use Public Key Authentication” in the More Options.
- Server: ftp-private.ddbj.nig.ac.jp
- Port: 22
- Username: (D-way Login ID)
- Password: (Leave empty)
- Add to Keychain: (Check)
By default, the private key is created in “User’s home folder > .ssh folder (invisible in Finder) > id_rsa”.
At the first time of login, a warning message is displayed; however, please select “Always” (this message will not be displayed again).
After login successfully, your private directory in the server is displayed in the window. Select the files in your PC and drag & drop them into the window to transfer the files to the server.
Users can ssh login ftp-private.ddbj.nig.ac.jp server by using a private key. Executable commands are restricted to the following ones. Users can delete unnecessary files.
ls cd cp mv rm more mkdir tar gzip gunzip bzip2 bunzip2 zip unzip
Set the hold date within four years or choose immediate release when processed. Submitters’ name and affiliation will be public but e-mail address will not be disclosed.
You can delete an un-submitted GEA submission at “Delete submission”.
Select a submitted project registered in your account. If a BioProject is not registered, please go to the BioProject submission site and submit a project.
To use a project obtained in the other account, please contact GEA team.
Select submitted BioSamples registered in your account. If BioSamples are not registered, please go to the BioSample submission site and submit samples.
To use samples obtained in the other account, please contact GEA team.
Enter information for IDF (Investigation Description Format).
- Protocol: Pre-checked protocols are mandatory.
- Publication: Describe associated publications by PubMed ID or DOI. For unpublished manuscript, please inform us the publication ID after assignment.
- Array Design: When an array design is available in ArrayExpress/GEA, enter an array design accession number “A-XXXX-n”. When an array design is not available, register a new array design by uploading an array design file into the GEA submission directory.
- Data File Type: Raw and processed data files are required for microarray experiment submission. We strongly recommend to submitting raw and processed data file per sample. Accepted Data Files Formats for microarray experiment.
Enter information for SDRF (Sample and Data Relationship Format).
- Name columns and attribute columns for Source Name: Generated from BioSamples.
- SDRF rows: 1 row for 1 BioSample.
- Protocols: Protocols described in IDF are inserted to appropriate positions of SDRF with temporary protocol IDs (e.g., ESUB000350_Protocol_1)
- Technology Type: “array assay” for microarray submission.
- Array Design REF: array design accession or filename described in IDF.
Enter required fields by overwriting <Required: fill in the content> tags.
Fields you need to add.
- Material Type: Enter controlled terms.
- total RNA
- polyA RNA
- cytoplasmic RNA
- nuclear RNA
- genomic DNA
- Label: Enter label compounds used to label the extracted molecule such as biotin, Cy3 and Cy5.
- Array Data File and Comment[Array Data File md5]: Enter filename and md5 checksum pair for each raw data file.
- Derived Array Data File and Comment[Derived Array Data File md5]: Enter filename and md5 checksum pair for each processed data file.
- A list of filename and its md5 checksum (output of md5sum command) can be provided as a file <GEA submission ID>.md5 (e.g., ESUB000001.md5) (when the checksum values are provided in both SDRF and .md5 file, those in the .md5 are used).
- Factor Value[enter experiment factor name here]: A user-defined name for each experimental factor studied by the experiment. These experimental factors represent the variables within the investigation (e.g. growth condition, genotype, organism part). The actual values of these variables will be listed in the “Factor Value ” columns. Example:
- Factor Value[strain]
Select the entered SDRF file and continue.
Overview and submit
You can download the IDF and SDRF files and check them. When correction is necessary, go back to the previous tab and corrent metadata.
Submit the IDF and SDRF metadata by clicking the “Submit” button.
When data files described in the IDF and SDRF are not found in the submission directory, an error message “Data file is not uploaded” is shown and the submission is aborted.
The validator checks submitted IDF and SDRF files according to the validation rules and gives warning and error messages. Errors need to be resolved for submission.
GEA accession numbers are issued to completed GEA experiment.
You can allow reviewers access to private records by communicating a reviewer accesss token.
Update in each database
|Annotated sequence database||Request updates from web form|
|Sequence Read Archive (DRA)||Login D-way and update metadata
To add or delete sequencing data, request updates from web form
|Genomic Expression Archive (GEA)||Request updates from web form|
|BioProject/BioSample||Request updates from web form|
Withdraw archived objects
To withdrawing archived Experiment, please contact us.
MD5 (Message Digest Algorithm 5) is a hash function which calculates a hash value (MD5 number, 32-digit numbers and letters) of a given file. Because the MD5 number of the damaged file is distinct from the original one, we can check whether the transferred file is intact or not by comparing the numbers before and after the file transfer.
Obtain the MD5 numbers of the files by executing,
$ md5sum file1 file2 9F6E6800CFAE7749EB6C486619254B9C file1 B636E0063E29709B6082F324C76D0911 file2
Obtain the MD5 numbers of the files by executing,
$ md5 file1 file2 9F6E6800CFAE7749EB6C486619254B9C file1 B636E0063E29709B6082F324C76D0911 file2
Install and run the Fsum Frontend (sourceforge.net/projects/fsumfe/).
At first, tick off “md5”.
After clicking the [+] button, open the sequence data files that you need. You can select multiple files at the same time.
Click the [Calculate hashes] button. The MD5 numbers of the files are displayed.
By clicking the [Export] button, you can obtain the list of the MD5 numbers as a html, a csv, or a xml file.