{"id":78,"date":"2019-05-26T16:24:40","date_gmt":"2019-05-26T16:24:40","guid":{"rendered":"https:\/\/beta.research.ece.ncsu.edu\/ivmcl\/?page_id=78"},"modified":"2026-10-03T23:36:47","modified_gmt":"2026-10-03T23:36:47","slug":"ivmcl","status":"publish","type":"page","link":"https:\/\/research.ece.ncsu.edu\/ivmcl\/","title":{"rendered":"Home"},"content":{"rendered":"\n
<\/div>\n\n\n\n
\n

Intelligent systems work by putting parts together. We study the three operations every such system performs: how it composes<\/strong> parts into wholes, selects<\/strong> among alternatives, and remembers<\/strong> what it has learned. Our tools are grammars of parts and alternatives (And-Or graphs) and learning methods that infer such structure from data. Our goal is coherent computing via grammar-typed modeling<\/strong>: models whose composition, selection and memory stay reliable as they scale and as their tasks grow longer.<\/p>\n\n\n\n

Our long-term research goal is to pursue continually explainable and robust computing systems that can ALTER<\/strong> (Ask, Learn, Test, Explain and Refine) for AIGCGT<\/strong> (AI Generated Content and Ground-Truth).<\/p>\n\n\n\n

The iVMCL<\/strong> laboratory in ECE at NC State is led by Dr. Tianfu (Matt) Wu<\/strong>, Associate Professor of Electrical and Computer Engineering, and is affiliated with the Visual Narrative Initiative.<\/p>\n\n\n\n

\n
Our research \u2192<\/a><\/div>\n\n\n
Join us \u2192<\/a><\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n
\n \"Overview
Compositional and grammar-guided models<\/strong>: Can the structure of a model (which parts, which connections, which groupings) be learned from data instead of fixed by hand?<\/div><\/a>\n \"Overview
PaCa-ViT at a glance<\/strong>: image patches attend to a few learned clusters, whose heatmaps can be inspected directly.<\/div><\/a>\n \"Overview
Structured geometry: wireframes, planes and 3D<\/strong>: How can a network that predicts dense pixel maps reliably recover sparse geometric structure, such as line segments, wireframes, planes and 3D boxes, from images?<\/div><\/a>\n \"Overview
MvACon at a glance<\/strong>: cluster-based context added to 2D features before lifting improves query-based multi-view 3D detectors.<\/div><\/a>\n \"Overview
NEAT at a glance<\/strong>: the attraction field lifted into a 3D neural field reconstructs 3D wireframes without cross-view matching.<\/div><\/a>\n \"Overview
ScaleLSD at a glance<\/strong>: self-supervised line detection trained on about 10 million unlabeled images.<\/div><\/a>\n \"Overview
LoGo-CAP<\/strong> (CVPR 2022): multi-person pose estimation in which the joints of each person refine one another’s locations with local and global context<\/div><\/a>\n \"Overview
Face3D<\/strong> (ECCV 2016): face detection that builds a 3D mean face model into the ConvNet, so face proposals come from estimated 3D pose and key-points<\/div><\/a>\n \"Overview
Controllable generation<\/strong>: How can a generator be told where each object goes and what each one looks like, so a user can move, relabel or restyle one object while the rest of the image stays fixed?<\/div><\/a>\n \"Overview
Adversarial attack and robustness<\/strong>: How much control can a tiny, carefully chosen change to an image give an attacker over a classifier, down to dictating its whole ranked list of top predictions, and what do such attacks reveal about what the model has learned?<\/div><\/a>\n \"Overview
Graph neural networks and self-supervised learning<\/strong>: How can a graph neural network learn useful node representations when the edges cannot be fully trusted, because connected nodes are often unlike each other or the observed edges are noisy and incomplete?<\/div><\/a>\n \"Overview
HarmonyGNNs at a glance<\/strong>: each node decides how much to trust its neighborhood, learned without labels.<\/div><\/a>\n \"Overview
Continual and efficient adaptation<\/strong>: When a network must learn new tasks one after another, how should it decide which parts to reuse, which to adjust cheaply and which to add, without forgetting earlier tasks or wasting parameters?<\/div><\/a>\n \"Overview
WeGeFT at a glance<\/strong>: fine-tuning updates generated from the pretrained weights, with very few trainable parameters.<\/div><\/a>\n<\/div>\n\n\n\n

Research at a glance<\/h2>\n\n\n\n
\n
\n
\"Overview<\/figure>\n\n\n\n

Compositional and grammar-guided models<\/a><\/h3>\n\n\n\n

Compose<\/span> Can the structure of a model (which parts, which connections, which groupings) be learned from data instead of fixed by hand?<\/p>\n\n\n\n

Read more \u2192<\/a><\/p>\n<\/div>\n\n\n\n

\n
\"Overview<\/figure>\n\n\n\n

Structured geometry: wireframes, planes and 3D<\/a><\/h3>\n\n\n\n

Compose<\/span> How can a network that predicts dense pixel maps reliably recover sparse geometric structure, such as line segments, wireframes, planes and 3D boxes, from images?<\/p>\n\n\n\n

Read more \u2192<\/a><\/p>\n<\/div>\n\n\n\n

\n
\"Overview<\/figure>\n\n\n\n

Controllable generation<\/a><\/h3>\n\n\n\n

Compose<\/span> How can a generator be told where each object goes and what each one looks like, so a user can move, relabel or restyle one object while the rest of the image stays fixed?<\/p>\n\n\n\n

Read more \u2192<\/a><\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\"Overview<\/figure>\n\n\n\n

Adversarial attack and robustness<\/a><\/h3>\n\n\n\n

Select<\/span> How much control can a tiny, carefully chosen change to an image give an attacker over a classifier, down to dictating its whole ranked list of top predictions, and what do such attacks reveal about what the model has learned?<\/p>\n\n\n\n

Read more \u2192<\/a><\/p>\n<\/div>\n\n\n\n

\n
\"Overview<\/figure>\n\n\n\n

Graph neural networks and self-supervised learning<\/a><\/h3>\n\n\n\n

Select<\/span> How can a graph neural network learn useful node representations when the edges cannot be fully trusted, because connected nodes are often unlike each other or the observed edges are noisy and incomplete?<\/p>\n\n\n\n

Read more \u2192<\/a><\/p>\n<\/div>\n\n\n\n

\n
\"Overview<\/figure>\n\n\n\n

Continual and efficient adaptation<\/a><\/h3>\n\n\n\n

Remember<\/span> When a network must learn new tasks one after another, how should it decide which parts to reuse, which to adjust cheaply and which to add, without forgetting earlier tasks or wasting parameters?<\/p>\n\n\n\n

Read more \u2192<\/a><\/p>\n<\/div>\n<\/div>\n\n\n\n

Meet the team<\/h2>\n\n\n\n
\n
\n
\"Portrait
Tianfu (Matt) Wu<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"Portrait
Chinmay Savadikar<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"Portrait
Zhendong Liu<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"Portrait
Gabriela Goldner<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"Portrait
Vidit Ghuliani<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

Members and alumni \u2192<\/a><\/p>\n\n\n\n

Our research is supported by ARO, NSF, IARPA, the Laboratory for Analytic Sciences (LAS), Sony and Salesforce (details<\/a>).<\/p>\n\n","protected":false},"excerpt":{"rendered":"

Intelligent systems work by putting parts together. We study the three operations every such system performs: how it composes parts into wholes, selects among alternatives,…<\/p>\n","protected":false},"author":92,"featured_media":0,"parent":0,"menu_order":-1,"comment_status":"closed","ping_status":"closed","template":"page-landing.php","meta":{"_acf_changed":false,"footnotes":""},"class_list":["post-78","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/pages\/78","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/users\/92"}],"replies":[{"embeddable":true,"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/comments?post=78"}],"version-history":[{"count":61,"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/pages\/78\/revisions"}],"predecessor-version":[{"id":663,"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/pages\/78\/revisions\/663"}],"wp:attachment":[{"href":"https:\/\/research.ece.ncsu.edu\/ivmcl\/wp-json\/wp\/v2\/media?parent=78"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}