Server move & code update

The server has been migrated; hard links using 130.60.44.174 will point to an outdated version! Please use only www.progenetix.net.

Though not very apparent, he codebase has been changed a lot. This is mostly in the "Tools" section. However, some tools are not completed yet; e.g. one cannot directly load ISCN annotated files for visualization, or process aCGH data matrices. In "emergency" cases, you may email me. Segment lists should work fine; so pls. check the output options of your aCGH software first.

Happy Holodays!

Michael.

R code snippets

As a result of moving the graphics etc. mostly to R, we have started to post code snippets.

End of progenetix@googlegroups

We have closed down progenetix.googlegroups.com due to spamming/maintenance issues. The site has now a locally hosted WiKi which currently only allows posts by administrators; so please email me directly with issues.

One FISH, two FISH, red FISH, blue FISH ...

... is:

A) the title of one of the "Dr. Seuss" books (you would know if you have lived with children in pre-school age in the U.S.)
B) an editorial aricle by Michelle LeBeau

To not stick too much to the green=gain | red=loss color scheme (compared to the strange but widely used red=high expression etc. scheme of the expression array community), today two radical changes took place:

1. yellow is the new green, blue is the new red (therefore accommodating people with color recognition problems, though we already did refrain from pure red/green ...)
2. ideograms are plotted by new routines, implemented in R: larger chromosomes, better level lines, PNG + PDF vector graphics (after the links); however, no single band links to USCC anymore (you either know how to get there, or you want a data analysis collaboration).

Enjoy,

Michael.

The end of our metaphase banding data ...

Today we have published an update to the Progenetix database, which does not contain any data from metaphase analyses. However, we will keep the publication data online (e.g. mostly Mitelman references, with some updates/addtl. information). Users interested in mining data from banding analyses may contact me directly.

Overall, this reduces the total number of cases slightly, to now 21436 (18291 CGH, 3217 aCGH) from 763 chromosomal and array CGH publications.

Article access

For all people with database registration, access to all the PDFs (>1500) has been enabled. Authentication is required, and this is considered as personal information sharing among collaborators ...

GEO data sets

We have started to re-evaluate public aCGH data sets, available as raw or Log2 data from NCBI's GEO. Since the data there exists in a heterogeneous (MIAME-compatible) mess of data formats, GEO data will be made accessible either as locus-mapped (normalized) probe data, or as segmentation output files. For a start, we have posted some data sets from Affymetrix genome array experiments, for which the raw data was re-analyzed using the aroma.affymetrix package with custom plotting routines. Please browse the aCGH publications for the corresponding icon.

Progenetix@GoogleGroups

I have created a group site at GoogleGroups:

http://groups.google.com/group/progenetix/

Comments, site updates etc. will now be discussed there; please subscribe!

And the current tally is:

• 21256 cases from 777 publications


• 17409 CGH


• 2674 array CGH

Mitelman database publication update

As reference for tumor karyotype analyses using traditional cytogenetic techniques, the literature database has been updated with the current content of the Mitelman database. The listing references now to 3207 publications from that source, representing 58197 cases. The vast majority of these cases are not included in the Progenetix database, but can be visualized through the article links.

Change in ProgenetiXML format

For more flexible analysis options, the ProgenetiXML format now has separate fields for the Golden Path mapped status information for CGH, aCGH and banding data.

> 20,000 individual oncogenomic profiles

By incorporation of some large aCGH studies, today the Progenetix database jumped the 20k mark and contains now 20122 cases from 682 publications:

• 17154 of 31614 found CGH experiments


• 1926 of 10814 found aCGH experiments


• 1787 metaphase banding data sets

Again, above numbers include some overlaps.

19013 cases ...

The database has reached >19000 cases today:

• 16542 CGH


• 1503 aCGH


• 1757 banding/MFISH/SKY (mostly cell lines)

Also, the website tools (e.g. aCGH to ProgenetiXML) have seen some overhaul, making them more robust and introducing new features (high-resolution ideograms, segments file input etc.).

Enjoy!

Improved search options

Search options have been improved, with free text search now used in "Search Entities, Loci and Analysis Groups". As example:

lymphoma, NHL, lymph node, leukemia, hemato

will return all ICD entities, ICD loci and analysis groups with one of the comma-separated values in the title. From here, you can display each single subset by clicking on the ideogram button, or select several of the subsets for further filtering & analysis (registered users only).

Jumping 18k

Thanks to some data submissions, and notably today's articles from Itziar Salaverria Frigola (lab of Elias Campo), Progenetix has finally jumped the 18000 cases mark. The lack of ost of the array CGH data is still somewhat puzzling; either case specific data is not shown at all, or "raw data" is dumped in GEO | ArrayExpress | SMD - where nobody but groups with dedicated bioinformaticians will get it out. Probably, I will have to work on that ...

Enjoy the data, especially the expanded new B-NHL content!

Search update

The future is here (according to last post): For registered users, the pre selected publications can now be used to further limit the selection & analyze the cases ...

Feedback, please.

Search implementation

Instead of fixed listings for publications containing CGH/aCGH/banding analyses, a search engine for articles has been implemented. Options are:

• Autor names


• words from the title


• techniques


• array parameters for aCGH

Please comment on the features. Future extensions probably will include specific case search based on the pre-selected publications ...

Minor graphics changes

The imbalance histograms have been scaled to increase readability. Also, some broken links have been fixed.

Speed

The active database mining has been sped up (by introducing an indexing system). For registered users, when using:

Tools & Source => Source data selection, download and analysis => Use Progenetix Database

the options for data selection will load almost instantaneously.

Enjoy!

ProgenetiXML format enhancement: Golden Path

I have updated the ProgenetiXML format. The main improvement is the addition of a Golden Path aberration annotation:

<GPANNOTATION>chr7:0-158628139:1::chr8:0-146274826:1::chr10:70300000-135413628:-1::chr12:0-132449811:1::chr13:16000000-114142980:-1</GPANNOTATION>

The GP-annotation represents a transcription of the chromosomal band status. For data derived from aCGH experiments, this may lead to an expansion of imbalances, since originally every band containing a full or partial gain/loss is considered as imbalanced. This may be changed in the future, with direct usage of the original GP array data, where available.

Additional changes:

• removal of extraneous XML tags, for the cases lacking data in those entries


• addition of a full data matrix (862 bands status and all annotations) as download option prior to visualization

Enjoy,

Michael.

improved color scheme

since pure red-green works bad for some people, the histo and ideogram colors have been slightly changed (yellowish red, blueish green). i am happy about further suggestions!

new server

sometime late january, the site has been moved to a new server (Mac Pro, Dual 2.8GHz, 8GB RAM). this should make the site more user-friendly ...

but: do not use deep links or bookmarks - stick to www.progenetix.net!

and, please, do report problems.

ISCN parser modifications

Thanks to a review of selected cases by Turid Knutsen, a number of changes have been implemented in the karyotype parser portion of the ISCN2matrix engine. This concerns mostly the handling of ambiguous annotations, but also some previous omissions of specific patterns.

The changes are active in the online parser, and will be propagated to the data set soon.

data review and site overhaul

The site has had a layout overhaul (the changes in code are more than skin deep, though). On the data side, the emphasis is mostly on review and adding clinical information. As a ballpark estimate, gender and age information should be available for about half the cases - but this may take some months. Even followup / survival data will become accessible for a proportion of cases. Of course, this data is dependent on the published results, and / or the willingness of the authors to dig deeply into their old lab records ...

On the minus side, the case number just dropped slightly, to now 16998. This was due to one doubled publication entry, and one pure SKY data set that was removed (still accessible through NCBI).

As general policy, metaphase banding (incl. MFISH/SKY data) is only presented if it had been directly added to Progenetix (e.g. the DSMZ cell lines, or as part of some specific data review project). Otherwise, only cases with composite analysis (e.g. CGH and banding) are kept for now.

Bug fix

The latest editions had a bug, where for the publication pages the histograms and numeric values were showing "0" aberrations, for every band. Ideograms were not affected. This has been fixed on 20071119.

Layout changes and data update

The site underwent some layout and option changes. Most interestingly, the publication information now links to the Mitelman database if the publication is included there. This is mostly the case for the articles listed in the "bandingtracker", and indicated by the icon. The data itself is not stored on the Progenetix server, but rather accessed from the CGAP server.

Current Progenetix data content:

• 17023 experiments from 667 publications


• CGH: 15163 experiments, 571 articles


• array CGH: 827 experiments, 30 articles


• banding/SKY: 1017 experiments, 72 articles

brain tumor data update

Thanks to Ruthild Weber, the brain tumor data (glia tumors, meningiomas) has been expanded; some other data (oral SCC, hepatoblastomas) was expanded as well.

Current stats:

• 16252 cases from 634 publications


• 14174 CGH


• 533 array CGH


• 1055 metaphase banding and 16 SKY


• 474 of those cases were analyzed with several techniques (possibly more, but for those the data is combined

data update, presentation

After some interlude, a number of CGH and aCGH publications have been added. The total number of cases now is 16073, including 13995 analyzed by CGH and 533 analyzed by array CGH. Also, the presentation from last week's MC-GARD meeting (overview about genomic profiles from 5918 malignant epithelial neoplasias = carcinomas) has been put online (link on homepage). Enjoy!

XML changes and analysis tool update

In the ProgenetiXML format, the case id label has been consistantly renamed CASEID. Also, TRDEATH has been renamed DEATH (tumor related death would be rarely different in information content; maybe another field will be added to flag it). Consequently, TTRDEATH is now FOLLOWUP. Pls. let me know, if you discover broken parsing.

The analysis tools have been improved, adding automatic clinical subset generation. Cases are assigned to a unique clinico-patholocical subset (e.g. Carcinoma: breast; HNSCC ...), based on ICD code and locus. Also, an overview heatmap of aberrations of the subsets is generated from percents of gains and losses (well, try it out ...).

clinical data

For some of the cases, clinical data is now available through the source download option. This is spotty at best, and hopefully will improve over time; however, it may provide interesting information, in some instances.
Most frequently, age/gender/grade/stage will be found. Survival/recurrence data is rare. "TRDEATH" and "TTRDEATH" also include sometimes cases with non-tumor related death; I probably will rename the fields later (one may infer a non-tumor death from "RECURR = 0", if annotated).

2007 February update

All of the currently 15765 cases from 618 publications have been reviewed regarding diagnosis codes, and the annotations of the original diagnoses have been streamlined.

CGH 13746 cases
array CGH 451 cases
composite 462 cases
banding 1055 cases

aCGH2ProgenetiXML: array CGH / matrix CGH "rev ish" data conversion

After developing custom aCGH to "rev ish" conversion scripts for each aCGH data table I could get hold of, I have developed a basic online application. The aCGH2ProgenetiXML conversion script lets you move your array CGh or matrix CGH data tables to the ProgenetiXML format, generating 862 bands based chromosomal annotation (ISCN 1995 rev ish), ideogram etc.

You can use log ratio or status data, and adjust to the data by using case specific or general thresholds. Please drop me a note for features/complaints.

Enjoy http://www.progenetix.de/~pgscripts/cgi-bin/front_aCGH2ProgenetiXML.cgi (address subject to change, as always use www.progeentix.net, and follow the links).

Updates of tools and appearance

There has been an ongoing refinement of the data analysis tools, which should make them more user-friendly: enjoy!
Also, CGH ideograms have gotten some background shading for regions with sometimes ambiguous CGH results (centromers, 1ptel ...); the same bands can be deselected from the analysis, when generating heatmaps etc.
As for the content: no new cases right now, but there has been a revisiting of some of the ICD codes (e.g. in breast carcinomas).

Search and analysis features for registered users

For all registered users, I have added the option to select, visualize and analyze chromosomal imbalance data directly from the Progenetix data set:

1. go to http://www.progenetix.de/~pgscripts/cgi-bin/front_armcluster.cgi
2. go down to "Data Access (registered users)"
3. use your registered email + password

After a short interval, you will be presented with the option to select specific entities/loci/techniques. You can select in any of the categories, using multiple selects.

Example:

--------
8140/0
8140/3
8140/3
8144/3
--------
C15
C16
--------
CGH
aCGH
composite
composite aCGH
--------

... would result in 497 of 15422 cases, which could then be downloaded, or used for ideogram generation, clustering etc.

PLEASE DO NOT SELECT VERY LARGE SUBSETS for online analysis; some of the downstream tools choke at about >1000 cases.

I would be glad to receive some feedback and suggestions! And, as always, please provide your published (a)CGH data.

Web site code update

The layout of the Progenetix site has been slightly refreshed, with a general information column now appearing on all pages. Additionally, the HTML code has been changed to a fixed-width, CSS based layout: a screen resolution of 1024x768 is now a virtual necessity.
15422 cases from 609 publications.

Database update and error fix

15327 cases from 607 publications (incl. 399 aCGH cases and 13361 CGH, as well as 393 "composite" analysis results).

A bug in the data of Alcock et al. (PMID 12800148) has ben fixed - the data listedaddtl. non-existant changes added during copying.

CGHtracker, aCGHtracker and bandingtracker ...

The Progenetix website now unifies the literature overview for different (molecular-) cytogenetic screening techniques:

• 924 chromosomal CGH publications
• 130 array CGH publications
• 8022 publications with metaphase banding/SKY/MFISH data (mostly linked to the Mitelman database)

All these publications are linked to their respective PubMed entries; Metaphase banding publications are addtl. linked to the Mitelman database. For more than 50% of the CGH cases (13238) as well as for >400 aCGH and some banding/SKY/MFISH data, source and ideograms are available through Progenetix itself. 

SKY data update and removal of breakpoints

The SKY results for the NCBI cases have been added, resulting in generation of composite imbalance profiles. 
As a reminder: Imbalances are are generated from banding/SKY/MFISH (they are all considered the same quality) first; then, everything found in CGH is added. This seems to be reasonable, as CGH should be better for imbalances, though may miss some changes just appearing in subclones.
However, I have decided to remove the blue breakpoint-bars from the ideograms, as they tended to irritate some users. The data is still available in the downloads, though 

14795 cases from 577publications

For the update today, I have had a look at the current state of the NCBI SKY/CGH database. I (re-) wrote a parser for their XML file format, which currently only interprets CGH annotations.
Of the 840 human CGH cases from the current NCBI Sky/CGH XML dump, about 600 were included into th edatabase from this source; the remaining cases had been collected from the literature or submitted by the authors to Progenetix.
For cases with both SKY and CGH data, the MFISH/SKY information will be added later.

Internet Explorer 6 issues

Checking the Progenetix site in IE 6 after a long time, I have realized the incorrect rendering of some pages. Basically, the size attribute of the parent div is not inherited, leading to table cells being stretched off screen. Therefore, the last column with the ideogram link is only shown when scrolling horizontally.
As said before, use Firefox or Safari, or Konqueror or ... but not Internet Explorer. Really.
For now, I move the link column to the leftmost position :-(

Change of the XML format

The progenetiXML format has been expanded. Karyotype annotations are now separated according to their technique, in KARYO_CGH, KARYO_ACGH and KARYO_BANDING.The change was introduced to better handle cases analyzed with several techniques. For older XML files, KARYO + TECHNIQUE will result in correct interpretation. Please be aware of the changed template for tab-delimited files, too (linked on the ISCN2matrix page).

New input and analysis options

The analysis software has now some options for clinical data integration, e.g. interval specific Kaplan-Meier plots. For generating the proper Progenetix XML input files, the ISCN2matrix converter allows now the upload of complex tab-delimited files; a template is provided.

CGH and array CGH literature overview now online

After some beta'ing under diverse addresses, I have posted two pages containing separate lists for all CGH and array CGH (matrix CGH) publications reporting whole genome screens (that is, no "2p tiling path" etc.). I ill try to keep the lists current, and to include additional information, esp. regarding array CGH (array type - BAC, cDNA, oligo; resolution; source data links). Publications with cases in Progenetix are linked to the data there. Enjoy and comment, please!

Database zombies and broken links ...

As Richard Birnie pointed out, there was a zombie of the website when going directly to http://www.progenetix.net/progenetix/Aboutprogenetix.html

This old mirror has been removed.

Due to the frequent server moves, links got broken; so the best is to stick to the domain names:

www.progenetix.net
www.progenetix.com
www.progenetix.de
new:
www.progenetix.eu

Additional download option

The raw data of the "Misc. Groups" is now available for download, too.

Fix of scoring bug

For a while the band score engine was displaying high scores for gains in the losses section, too; this has been fixed.

Due to a typo, many links pointing to CGIs (e.g. source download, ISCN2matrix converter) were broken. Fixed.
Also, I finally fixed the CSS, so now everybody should be able to enjoy the page in sans-serif (Helvetica Neue Light if you have a Mac ...).

Termination of progenetix.ufscc.ufl.edu

The following server addresses are not valid anymore; any links to them won't work:

progenetix.ufscc.ufl.edu
pirx.ufscc.ufl.edu
159.178.64.62

Please report broken links if encountered in the current pages, to contact@progenetix.net

News and History moved ... again

Due to the server move, I also have switched the News and History blog to a blogger.com based hosting. The direct address is http://progenetix.blogspot.com. Link forwarding should be in place soon, too. The old news can be found at 
www.progenetix.net/progenetix/Newsandhistory2005_2006.html

Progenetix Server Migration

The Progenetix server is, again, in the process of being migrated. The server IP is 217.160.22.58. Scripts (e.g. data export, analysis, ISCN2matrix converter) may run under 159.178.64.62 (progenetix.ufscc.ufl.edu) for a while; please report broken links!