• about
  • resources
  • events
  • contribute
  • republishing
    • ☰
  • news
  • articles
  • opinions
  • education

    • ΑΙhub.org
     

    Tweets from #ICLR2019


    by
    16 May 2019


    share this:

    ICRL, the International Conference on Learning Representations, was held May 6th to 9th 2019 in New Orleans.

    Relive the conference through some of the top tweets (#ICLR2019).

    Invited talks

    Best Papers

    Congratulations to the two ICLR 2019 Best Paper winners!

    The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (arXiV)
    Jonathan Frankle · Michael Carbin

    Abstract - Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance.

    We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively. Based on these results, we articulate the "lottery ticket hypothesis:" dense, randomly-initialized, feed-forward networks contain subnetworks ("winning tickets") that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective.
    We present an algorithm to identify winning tickets and a series of experiments that support the lottery ticket hypothesis and the importance of these fortuitous initializations. We consistently find winning tickets that are less than 10-20% of the size of several fully-connected and convolutional feed-forward architectures for MNIST and CIFAR10. Above this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy.

    Summary in MIT Tech Review.

    Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks (arXiv)
    Yikang Shen · Shawn Tan · Alessandro Sordoni · Aaron Courville

    Abstract - Natural language is hierarchically structured: smaller units (e.g., phrases) are nested within larger units (e.g., clauses). When a larger constituent ends, all of the smaller constituents that are nested within it must also be closed. While the standard LSTM architecture allows different neurons to track information at different time scales, it does not have an explicit bias towards modeling a hierarchy of constituents. This paper proposes to add such an inductive bias by ordering the neurons; a vector of master input and forget gates ensures that when a given neuron is updated, all the neurons that follow it in the ordering are also updated. Our novel recurrent architecture, ordered neurons LSTM (ON-LSTM), achieves good performance on four different tasks: language modeling, unsupervised parsing, targeted syntactic evaluation, and logical inference.

    And a summary tweet from Microsoft with an accessible blog post.

    Increasing diversity

    Online presentations

    38 presentations can be watched here.

    As well as debates.

    And here are a couple researchers putting their slides online.

    Other summaries and highlights from the conference

    AI for social good

    Also, what is going on here?

    Looks like a good PR move for their paper on the wizard of Wikipedia.

    That's a wrap! See you all next year!



    Sabine Hauert is Associate Professor at the University of Bristol, and Executive Trustee of AIhub.org
    Sabine Hauert is Associate Professor at the University of Bristol, and Executive Trustee of AIhub.org



                AIhub is supported by:

    Related posts :



    Interview with Eden Hartman: Investigating social choice problems
      24 Apr 2025
    Find out more about research presented at AAAI 2025.

    The Machine Ethics podcast: Co-design with Pinar Guvenc
      23 Apr 2025
    This episode, Ben chats to Pinar Guvenc about co-design, whether AI ready for society and society is ready for AI, what design is, co-creation with AI as a stakeholder, bias in design, small language models, and more.

    Why AI can’t take over creative writing
      22 Apr 2025
    A large language model tries to generate what a random person who had produced the previous text would produce.

    Interview with Amina Mević: Machine learning applied to semiconductor manufacturing
      17 Apr 2025
    Find out how Amina is using machine learning to develop an explainable multi-output virtual metrology system.

    Images of AI – between fiction and function
    and   16 Apr 2025
    “The currently pervasive images of AI make us look somewhere, at the cost of somewhere else.”

    Grace Wahba awarded the 2025 International Prize in Statistics
      16 Apr 2025
    Her contributions laid the foundation for modern statistical techniques that power machine learning algorithms such as gradient boosting and neural networks.

    Provably safe certification for machine learning models under adversarial attacks: Interview with Chen Feng
      15 Apr 2025
    Find out more about work presented at AAAI 2025.

    Repurposing protein folding models for generation with latent diffusion
      14 Apr 2025
    The awarding of the 2024 Nobel Prize to AlphaFold2 marks an important moment of recognition for the of AI role in biology. What comes next after protein folding?


    ↑
  • news
  • articles
  • opinions
  • education
  • about
  • resources
  • events
  • contribute
  • republishing
  • RSS
  • linkedin
  • facebook
  • twitter
  • Youtube

    • AIhub is supported by:






    ©2024 - Association for the Understanding of Artificial Intelligence


     










    ©2021 - ROBOTS Association