Submission to AGD

Submission Guide

The AGD accepts only de-identified data approved by NBDC. The users apply for data submission directly to the NBDC and the AGD will only accept submissions once the information of a successful application process has been passed from the NBDC to the AGD.

Following internal IDs are issued to metadata objects with prefixes.

Prefix Metadata object Digit Example
AGD_ Submission 11 AGD_00000000001
AGDS_ Study 11 AGDS_00000000001
AGDN_ Sample 11 AGDN_00000000001
AGDX_ Experiment 11 AGDX_00000000001
AGDR_ Data 11 AGDR_00000000001
AGDZ_ Analysis 11 AGDZ_00000000001
AGDD_ Data set 11 AGDD_00000000001
AGDP_ Policy 11 AGDP_00000000001

Sequence data file (raw/unaligned and aligned)

Supported AGD data submission formats are described below. If you have data in any other format or have any questions please contact the AGD.

Data files should be de-multiplexed prior to submission so that each data object is submitted with files containing data for a single sample only.

BAM format

Binary Alignment/Map files (BAM) represent one of the preferred AGD submission formats. BAM is a compressed version of the Sequence Alignment/Map (SAM) format (see SAMv1.pdf). BAM files can be decompressed to a human-readable text format (SAM) using SAM/BAM-specific utilities (e.g. samtools) and can contain unaligned sequences as well.
AGD strongly recommends to submit BAM including unaligned reads as primary data into Data.

The BAM files are nearly optimal in terms of compression and should be submitted uncompressed.

Fastq format

Primary sequence data submissions of single and paired reads are accepted as Fastq files that meet the following the requirements:

  • Quality scores must be in Phred scale. For example, quality scores from early Solexa pipelines must be converted to use this scale. Both ASCII and space delimitered decimal encoding of quality scores are supported. We will automatically detect the Phred quality offset of either 33 or 64.
  • No technical reads (adapters, linkers, barcodes) are allowed.
  • Single reads must be submitted using a single Fastq file.
  • Paired reads must split and submitted using two Fastq files. The read names must have a suffix identifying the first and second read from the pair, for example '/1' and '/2'; (regular expression for the reads "^(.*)([\\.|:|/|_])([12])$").
  • The first line for each read must start with '@'.
  • The base calls and quality scores must be separated by a line starting with '+'.
  • The Fastq files must be compressed using gzip or bzip2.
Example of Fastq file containing single reads: 
@read_name
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65
...
Example of Fastq file containing paired reads: 
@read_name/1
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65
@read_name/2
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65
...

SFF format

The recommended submission format for the 454 platform is SFF.

The SFF files are nearly optimal in terms of compression and should be submitted uncompressed.

Array based (Genotypes, SNP, Expression)

The AGD accepts processed data from all types of array based technologies, such as genotypes, gene expression, methylations, etc.

We are very flexible regarding these submissions and would accept all types of files affiliated to the array experiment into the AGD array.

We recommend that both raw data (CELs) and analysis files be submitted, which will enable potential users of the data to recreate your results cited in the paper as well as view the described results.

The AGD also archives any sample-associated phenotype information.

Variation data file

The AGD supports submissions of sequence variations in VCF format. The vcf files are accepted as secondary analysis object.

Other data files

Analysis can refer to multiple Data and Sample objects and summary data summarizing data files of these objects can be registered to the Analysis. Example analysis files are listed below.

  • A reference sequence GFF3 file assembled by using multiple sequence files in a objects.
  • A table file summarizes normalized array data files in Arrays.
  • A phenotype table file summarizes phenotype information of Samples.

Metadata

The AGD data model is created by extending the Sequence Read Archive's model. The AGD metadata are composed of the XML objects.

AGD XML schema: AGD xsd

Example: single policy.
Example: single policy.
Example: separate policies for control and case.
Example: separate policies for control and case.

Summary information for public display

When AGD submissions are released, following metadata objects will be public on the AGD and NBDC websites to display summary of submissions.

Submission:
All fields excluding submitters' contact information.
Study:
All fields.
Data set:
All fields, number of samples, and filesize.
Policy:
All fields.

Submission

The submission XML is used to describe the submission transaction. It contains contact details of the submitter.

XML schema: AGD.submission.xsd

Study

The study XML describes your study in detail. It contains a title, a study type and an abstract, as it would appear in the publication. After publication, the study can be updated to include the PubMed ID of the paper containing the submitted data.

XML schema: AGD.study.xsd

Sample

The sample XML describes each of the samples used in your submission. A sample generally corresponds to a participant. Whilst the mandatory fields are minimal we highly recommend to describe the sample in as much detail as possible.

XML schema: AGD.sample.xsd

Experiment

The experiment XML is used to describe each unique set of experimental setup details. This includes the library preparation details, sequencing and microarray platforms. The experiment XML can represent both sequencing and array experiments.

XML schema: AGD.experiment.xsd

Data

The data XML describes the data files and their relationship with the experiments. The data XML serves as a file container.

XML schema: AGD.data.xsd

Analysis

The Analysis XML can be used to submit analyzed and summarized data to JGA. Sample phenotype files can also be submitted. Variation data can be submitted as VCF files. Only one VCF file can be submitted in each analysis and the samples used within the VCF files must be associated with JGA Samples. Optimally the VCF file would be associated with an INSDC reference assembly and sequences either by using accessions or by using commonly used labels.

XML schema: AGD.analysis.xsd

Data set

The dataset XML describes the data files, defined by the Data XML and Analysis XML, that make up the dataset and links the collection of data files to a specified Policy (Data Access Agreement). Submitters do not need to create data set when single policy is associated with all data files. Please consider the number of datasets that your submission consists of, for example, a case control study is likely to consist of at least two datasets.

In general, policy is determined by informed consent with participants.

XML schema: AGD.dataset.xsd

Policy

The Policy XML describes the Data Access Agreement (DAA).

XML schema: AGD.policy.xsd

Analysis only submission

To submitting only aggregated data or data which are not suitable for Sample-Experiment-Data, these data can be submitted as Sample-Analysis instead of Sample-Experiment-Data.

One Analysis for each Sample
One Analysis for each Sample
Analysis data aggregating Samples
Analysis data aggregating Samples

To submitting Analysis only data, please contact JGA team.

Update submissions

To update your submissions, please contact to the AGD team.