— the global portal for all things SBML

Level 3 Package Proposal Voting Results: multi

The call for votes for the 'multi' package was issued on September 19 and closed on October 3, 2011. A total of 34 votes were cast. Six individuals were not members of the sbml-discuss or sbml-multi mailing lists, and one individual responded twice; removing these left 27 valid votes.

The outcome of this vote is accept.

This outcome is based on the final question of the survey, which had four mutually-exclusive choices: Accept (proposal addresses a need that SBML should cover, and the approach clearly follows the stated principles), Reject (proposal does not address a need that SBML should cover), Revise (approach either does not follow the stated principles, or there is insufficient information to tell if it does), or Abstain, (I cannot fully assess the proposal as given, or do not wish to state an opinion). The following graph presents the results for this question:

The voting for this package proposal used an updated format that requested additional information from voters. The responses for each of these additional questions are reported separately below. The comments written by respondents for the overall assessment question are provided at the end.

Question about UTILITY

The question was posed as follows: Utility: the package addresses a problem whose solution SBML users are likely to find useful. It had four possible answer choices: (a) Agree, (b) Disagree, (c) Insufficient info, and (d) Abstain.

The following graph shows the overall total responses:

The following are the individual comments written by respondents. The votes in particular are indicated in parentheses before the comments.

  • (Voted 'Accept') There are an increasing number software tools that support rule-based modeling, and a common exchange format would be useful, as would be the annotation features of SBML.
  • (Voted 'Accept') Several software tools support “multi” features, thus the exchange of “multi” models would be useful.
  • (Voted 'Accept') Many useful models, particularly those that involve polymerization, aggregation, or multi-site post-translational modifications cannot be specified in SBML. This package would help meet some of these needs.
  • (Voted 'Accept') The multi package will or should support exchange of rule-based models. There are now a number of software tools for rule-based modeling, including BioNetGen, DYNSTOC, RuleMonkey, NFsim, KaSim, SSC, Moleculizer/Smoldyn, SRSim, and others.


The question was posed as follows: Biological orientation: the package's overall aim is to support the description of biological processes and phenomena. It had four possible answer choices: (a) Agree, (b) Disagree, (c) Insufficient info, and (d) Abstain.

The following graph shows the overall total responses:

The following are the individual comments written by respondents. The votes in particular are indicated in parentheses before the comments.

  • (Voted 'Accept') The multi format is especially important for models of cell signaling systems, but also for models of metabolic networks etc.

Question about COHERENCE

The question was posed as follows: Coherence: the package extends SBML in a way that follows naturally from Level 3 Core and other packages. It had four possible answer choices: (a) Agree, (b) Disagree, (c) Insufficient info, and (d) Abstain.

The following graph shows the overall total responses:

The following are the individual comments written by respondents. The votes in particular are indicated in parentheses before the comments.

  • (Voted 'Disagree') 1) The SpeciesType class does not have the expected relationship to the Species class. A more appropriate name would be "EntityType", "ComponentType", etc. 2) Species are implicitly defined as entities that match each selector in a list. The algorithm that constructs species as the intersection of selectors is undefined (and presumably left to the individual software platforms that will eventually support Multi). An explicit, constructive approach to Species definition seems would be more in line with the nature of SBML and most of the existing rule-based languages. The motivation for the proposed species definition is unclear: it seems that a species could be defined by a single selector, in which each contained SpeciesType is fully specified.
  • (Voted 'Disagree') The terminology is confusing: I really think the specification should be using terminology similar to a de facto standard, be it bngl, kappa or something extensively used. It's specially important not to overload terms that the community already use with a different meaning. The class hierarchy is confusing as well which leads to a lot of redundancies. For example, the unbound state definition is something that could be eliminated if the class hierarchy made more sense. for the reaction product and reactant definition" instead of designing a whole new syntax for defining the reaction conditions we could reuse some of the same syntax that was used for the component definition.
  • (Voted 'Disagree') A number of choices have been made in the structure of the current proposed format that are confusing at best and may be problematic in other ways. For example, the semantics of the transformations specified by rules are not clear, nor are they explained. The use of Selectors implies that both reactants and products must pass these filters in order for a reaction to occur, but what defines how the instances themselves are transformed by the rule? Another difficult issue in the proposal is the way that Species are defined as sets of SpeciesTypeInstances, which are themselves sets of specific instances of species selected by Selectors. It takes quite a while to understand the structure, and there are some serious potential complications, one of which it is easy to construct internally inconsistent definitions. Also, it appears that complexes are often, if not always, defined implicitly in such declarations. There is no example of where a complex is explicitly defined and intialized by a single selector. Instead, complexes are defined implicitly by specifying that two SpeciesTypeInstances in a complex are bound - the software will be required to manage the assembly of complexes with the correct number and identity of binding partners. It seems like there is considerable opportunity for mischief here. Reactions are also confusing, because it seems that the number of reactants and products for binding and unbinding reactions are different from their counterparts in more standard languages. For example, a binding reaction specified in Multi will have two reactants and two products, whereas normally one thinks of there being one product. It is not obvious looking at the reaction what transformation is taking place: a state change reaction would look identical to a binding reaction, just the selector names pointed to would be different. A minor point is that I think the SpeciesType element is misnamed. These represent the basic objects in the systems. Binding sites of a molecule are not properly a type of species, rather, they are components of species. This nomenclature can be the source of some confusion.
  • (Voted 'Disagree') In the current SBML proposal specification of species through selectors in place of direct combination of speciesTypeInstances creates unnecessary difficulties for SBML L3 “multi” processing, and contradicts SBML basics. Consider, e.g. specification of ligand species at page 48: it contains the list of all ligand-containing selectors. What if there are too many of them? To specify a receptor-ligand complex, we need to separately specify bound ligand as a part of “ligand” species and a “bound receptor” as a part of “receptor species”. The species representing a complex does not exist anywhere – only two instances of species types “ligand” and “receptors” exist. The connection between them is deduced from two separate selectors: one selects bound ligand from a pool of all ligands, another selects bound receptor from a pool of all receptors. It creates problems with stoichiometry and assignment of initial amounts that has to be distributed among multiple species. 1.Species in a new proposal is composed of speciesTypeInstances each of which has an initialAmount and SelectorReferences, some of selectorReferences are empty (see page 48). The nested structure of initialAmounts contradicts Species definition, where species is a pool of identical things. Specification of species through speciesTypeInstances with zero amount creates unnecessary logical constructs. 2.Incorrect stoichiometry. In the ligand-receptor binding (page 50) we have 2 reactant species and two product species, for total 4 species involved. This is wrong: in biochemistry we have two reactant species and only one product species, as ligand-receptor complex is a single species. Again, it is a direct violation of SBML, where the result of binding two entities should be a single entity.
  • (Voted 'Disagree') Multi would extended SBML but I feel it's a transformative change, not an incremental change. Many rule-based models, for all intents and purposes, cannot be expressed in a traditional form, which is what SBML Level 2/3 is designed to handle. So, no. Multi models are not traditional models.
  • (Voted 'Abstain') Have not checked in detail, but what I have seen looks fine.
  • (Voted 'Disagree') multi model are fundamentally different from other models that can be currently encoded in SMBL.

Question about ORTHOGONALITY

The question was posed as follows: Orthogonality: within reason, the package does not duplicate the purpose or data captured by other packages. It had four possible answer choices: (a) Agree, (b) Disagree, (c) Insufficient info, and (d) Abstain.

The following graph shows the overall total responses:

There were no comments written by respondents to this question.

Question about overall assessment

If someone voted to revise or reject, we asked them to provide an explanation. The following are the explanations that people provided, together with the way they voted on the overall assessment question. (Note: one of the people who voted 'Revise' did not provide an explanation; conversely, one of the people who voted 'Accept' provided a comment unbidden.)

  • (Voted 'Revise') An SBML extension that describes rule-based models is clearly needed. This proposal is a good first step, however the accepted proposal should be clear, no more complex than necessary, and suit the needs of the major rule-based modeling approaches. The motivation for the layers of complexity in the proposal is unclear. Further examples that show the representation of existing languages (e.g. Kappa, BNGL, BioCham) within the Multi proposal would be extremely useful.
  • (Voted 'Revise') Before I express any further criticism, I want to thank Nicolas and Anika for doing a fantastic job with the current proposal. A lot of work has gone into it, and they have done the community a great service. I think the basic issue with the current proposal is that the rationale for many choices that have been made has not been clearly explained (perhaps because of lack of time). The discussions out of which the proposal was formed, took place several years ago now, and a number of choices that have been made by the principal authors in restructuring the proposal have not been discussed by the community, at least not publicly. I believe that much of the confusion that I and others have about the current proposal can be resolved by further discussion and possibly some restructuring of the proposal. I am open to the choices that have been made here, but I need to better understand why they were made and need further evidence of their soundness, both from a design point of view and in terms of the logic.
  • (Voted 'Revise') See my comments above for Coherence part.
  • (Voted 'Revise') There are some major limitations of this proposal that make it incapable to solve the need it intends to. For example, under this proposal, a species can be either a molecule (e.g. a protein) or part of a protein (e.g. a domain or a tyrosine). Then, selectors are be used to group species (protein, domain, tyrosine, etc.) to define a molecule. However, in order to generate all the essential selectors, all pairwise interactions in a model must be analyzed, which is impractical for large models. Another example: All pairwise interactions must be analyzed to collect all the binding sites, which is required when defining species. This also makes the current proposal unsuitable for large models. To use this proposal, a software must read all the species, analyze a whole model, and translate it into its program objects. Reading and translating are the tasks of a parser, but model analyzing is not.
  • (Voted 'Accept') I have serious concerns about the current proposal. In short, it needs to be significantly revised for it to be practical. However, as far as the intent goes, the proposal is on track. Something like multi is needed to facilitate exchange of rule-based models. The question I have is... why is multi development stuck? We have an XML-based format for exchanging models that is supported by a number of software tools. It works fine and it is significantly simpler than the proposed multi format.

Please use our issue tracking system for any questions or suggestions about this website. This page was last modified 02:32, 8 October 2011.